Model adaptation for autonomous trucking in right of way

ABSTRACT

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for monitoring a dedicated roadway the runs in parallel to a railroad. In some implementations, a system includes a central server, an interface, and sensors. The interface receives data from a railroad system that manages the railroad parallel to the dedicated roadway. The sensors are positioned in a fixed location relative to the dedicated roadway. Each sensor can detect vehicles in a first field of view on the dedicated roadway. For each detected vehicle, each sensor can generate sensor data based on the detected vehicle in the dedicated roadway and the data received at the interface. Each sensor can generate observational data and instruct the detected vehicle to switch to an enhanced processing mode. Each sensor can determine an action for the detected vehicle to take based on the generated observational data.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit of U.S. Provisional Application No.63/225,067, filed on Jul. 23, 2021, which is incorporated herein byreference.

TECHNICAL FIELD

This specification generally relates to road surveillance, and oneparticular implementation relates to monitoring a dedicated roadway theruns in parallel to a railroad.

BACKGROUND

Vehicles can travel on roadways, highways, and backroads to theirdestination. In many cases, a vehicle can travel along a road with othervehicles and is positioned behind the other vehicles, next to anothervehicle, or in front of another vehicle during its journey.Additionally, vehicles often move positions on the roadway byaccelerating, decelerating, or changing lanes. Given the number ofvehicles in any given section of road, and the changing speed andpositions of the vehicles, collecting and maintaining vehicle speed andposition data, and other vehicle data, is a complex and processingintensive task.

SUMMARY

The subject matter of this application is related to a system thatmonitors a dedicated roadway for autonomous vehicles, running alongrailroad rights of way, e.g., whether parallel to or in the place ofconventional railroad operations. Specifically, the system facilitatesaccess to, monitoring of, and safe navigation of the roadway, forautonomous vehicles, such as autonomous trucks. The system can charge atoll or other fee to autonomous trucks moving along the dedicatedroadway from a first point to a second point. The charged tolls can beused to generate revenue for the railroad operator, e.g., potentially ata higher operating margin than what the railroad is typically able tocharge for railway operation—without materially adversely impacting theexisting rail business. In this manner, introducing tolled autonomousfreight infrastructure can be accretive to the value of the railroadright of way.

For example, a railroad right of way may include the Lehigh Railwaylocated in Pennsylvania, which is a short-line railroad that covers 56track miles. The Lehigh Railway connects between the Reading BlueMountain and the Northern Railroad along the Susquehanna River. In somecases, the Lehigh Railway can run anywhere between ten to thirty trainsper day. However, this utilization can fluctuate. A parallel roadwaythat enables toll charging of autonomous trucks carrying goods that maynot otherwise travel on the Lehigh Railway unlocks an ancillary sourceof revenue, offsetting days when the railroad is underutilized.

The system described in this application can support the safe movementof autonomous trucks on rail rights of way and charge a toll forautonomous trucks to operate on the parallel-dedicated roadway. Bycharging a toll for autonomous trucks to move goods from point A topoint B alongside the railroad, the railroad operator can unlockincremental value at potentially accretive margins versus when operatingas a railroad alone. At the same time, within the autonomous truckingmarket, there are significant risks to deploying autonomous trucks onactive roadways due to safety issues, complexity risks, and operationalchallenges, to name a few examples. As such, by having a dedicated lanethat connects a key freight corridor and runs in parallel to a railroadright of way, a significant advantage exists for autonomous freightoperators for deploying trucks within a controlled operating environmentthat improves reliability, safety, and an ability for autonomous trucksto move goods commercially and at scale. As a result, by providing aparallel-dedicated lane for autonomous trucking, the system can convertlegacy underutilized railroad right of way assets into advanced freightcorridors that deliver right of way monetization and increase value forrailroad operators while at the same time, delivering improved andaccelerated deployment of autonomy for trucking fleets.

In some implementations, the system can incorporate sensors placed in alongitudinal manner along the parallel roadway for monitoring thevehicles, their position, their movement amongst other vehicles, and forcharging a toll on the vehicles for using the parallel roadway. Thesesensors can communicate with one another, communicate with one or moretrains on the railroads, communicate with the autonomous trucks, andcommunicate with a central server, to name a few examples. Each sensorhas their own field of view for monitoring a designated area of theparallel roadway and can be spaced at a predetermined distance apartfrom one another alongside the parallel roadway. The sensors themselvescan include a LIDAR system, high definition (HD) video cameras, weathermonitoring devices, a radar, a BLUETOOTH™ system, and a Wi-Fi system, toname a few examples.

The sensors can, for example, generate observations regarding roadactors, e.g., vehicles, objects, or people, traversing on the parallelroadway. The sensors can calculate other characteristics about vehiculartraffic, e.g., vehicle density per unit area or vehicle congestion,vehicle headway, and vehicle dynamics, each relating to vehicles on theparallel roadway. For example, the sensors can identify an object as theobject enters its field of view. Based on the identification of theobject, the sensors can further describe a location of the vehiclesalong the configured roadway, a speed of the vehicle, a relationship ofthe vehicle to another vehicle, e.g., vehicle headway describingdistance and time between two moving vehicles, and others, to name a fewexamples.

In some implementations, an autonomous vehicle can include an autonomoustruck that utilizes vehicular automation. Specifically, the autonomoustruck is capable of sensing its environment using a variety of sensors.These sensors can include, for example, cameras, RADAR, LIDAR, sonar,inertial measurement units, and other advanced control systems.

In order to make decisions about traversing roadways autonomously,autonomous trucks can include one or more machine-learning models thatproduce outputs based on input data provided from its own sensors. Thesemachine-learning models can be trained to produce likelihood of objectdetections, human detections, proximity of objects, detections of redlights, detections of green lights, clear roadways, congestion, andother examples. In response, the processing components onboard theautonomous trucks can analyze the outputs of the trainedmachine-learning models and can determine one or more actions for theautonomous truck to take, e.g., turn left, turn right, accelerate,decelerate, stop, etc.

However, the sensors onboard the autonomous trucks may not accuratelycapture events ongoing within the range of the parallel roadway. Forexamples, the onboard sensors may not be able to identify that a trainhas fallen on the parallel roadway a few miles ahead of the autonomoustruck's current position. In some examples, the onboard sensors may notbe able to view events ahead or behind its current position based onvehicles on the parallel roadway blocking its field of view. This can bean issue when these events may cause the autonomous trucks to change itsmovement pattern, e.g., adjust speed, change course, or avoid obstacles,to name a few examples. The on-board capabilities of the autonomousvehicles can be impacted by outside factors and may not functionreliably 100% of the time. The system described in this applicationseeks to alleviate these constraints by delivering supplementalcomplementing to the onboard capabilities of the autonomous trucksduring its traversal of the dedicated lanes of the parallel roadway,thereby improving reliability and mitigating the operational burden ofremotely monitoring and intervening in autonomous truck operations.

Specifically, the autonomous truck can enhance its thinking, so tospeak, when entering the parallel roadway and augment the trainedmachine-learning model processing with sensor data from not only its owninternal sensors but with sensor data from the external sensors. Saidanother way, the autonomous truck can gain a clearer understanding ofits operating environment by utilizing an enriched set of sensor datafrom both onboard sensors and the external sensors placed longitudinallyon the parallel roadway while driving on the parallel roadway. Theinformation from the sensors can define the operating environment, e.g.,an environment encompassing the parallel roadway and the parallelrailroad, and be supportive of the decision making for the autonomoustrucks.

The sensors monitoring the parallel roadway can provide sensor data tothe autonomous trucks as they traverse the parallel roadway. Theautonomous trucks can provide the received sensor data and/or thesupplemental sensor data to their trained machine-learning to produce anenhanced output that improves the decisions making for the autonomoustruck. The enhanced output can indicate a likely action for theautonomous truck to take while traversing the parallel roadway. In someimplementations, the sensors monitoring the parallel roadway can processsensor data and provide an action for the autonomous truck to take. Inthis case, the autonomous trucks can effectively enhance its trainedmachine-learning model with enriched sensor data while traversing theparallel roadway. Consequentially, the trained machine-learning modelcan improve its decision making capability and determine safer, moreinformed, and better guided actions for the autonomous truck to take totraverse the parallel roadway—actions that would otherwise be difficultto produce without the sensor data from the external sensors.

In one general aspect, a method is performed by one or more processors.The method includes: receiving, at an interface, data from a railroadsystem that manages a railroad running parallel to a dedicated roadway;detecting, by each sensor in a plurality of sensors positioned in afixed location relative to the dedicated roadway, one or more autonomousvehicles in a first field of view on the dedicated roadway, and for eachdetected autonomous vehicle: generates sensor data for the detectedautonomous vehicle based on the detected autonomous vehicle on thededicated roadway and the data received at the interface from therailroad system; generates observational data based on the generatedsensor data; instructs the detected autonomous vehicle to switch to anenhanced processing mode; determines an action for the detectedautonomous vehicle based on the generated observational data, the actionindicative of an action the autonomous vehicle should take whentraversing the dedicated roadway; and instructs the detected autonomousvehicle to traverse the dedicated roadway based on the determinedaction.

Other embodiments of this and other aspects of the disclosure includecorresponding systems, apparatus, and computer programs, configured toperform the actions of the methods, encoded on computer storage devices.A system of one or more computers can be so configured by virtue ofsoftware, firmware, hardware, or a combination of them installed on thesystem that in operation cause the system to perform the actions. One ormore computer programs can be so configured by virtue havinginstructions that, when executed by data processing apparatus, cause theapparatus to perform the actions.

The foregoing and other embodiments can each optionally include one ormore of the following features, alone or in combination. For example,one embodiment includes all the following features in combination.

In some implementations, the method includes: displaying, at theinterface, data related to the railroad that traverses in parallel tothe dedicated roadway and one or more trains traverse the railroad, thedata comprising a number of the one or more trains, a direction of theone or more trains traveling on the railroad, and a number of railroads.

In some implementations, the method includes, wherein the autonomousvehicles that traverse the dedicated roadway comprise autonomous trucks.

In some implementations, the method includes: acquiring, by theplurality of sensors, first sensor data of the autonomous vehiclestraversing the dedicated roadway; detecting, by the plurality ofsensors, an identity for each of the autonomous vehicles from the firstsensor data; from the identity for each of the autonomous vehicles,determining, by the plurality of sensors, that each of the autonomousvehicles have entered the dedicated roadway; and in response,transmitting, by the plurality of sensors, an indication to each of theautonomous vehicles to switch to the enhanced processing mode.

In some implementations, the method includes wherein the enhancedprocessing mode comprises (i) a setting for operating an autonomousvehicle using sensor data from the plurality of sensors and the sensordata onboard the autonomous vehicle or (ii) a setting in which anautonomous vehicle utilizes an enhanced trained machine-learning modelfor producing actions for traversing the dedicated roadway.

In some implementations, the method includes: acquiring, by theplurality of sensors, second sensor data of the autonomous vehiclestraversing the dedicated roadway; acquiring, by the plurality ofsensors, third sensor data of one or more trains traversing the railroadthat traverses in parallel to the dedicated roadway; transmitting, theplurality of sensors, the acquired second and third sensor data to acentral server; receiving, by the central server, the second and thirdsensor data from each sensor of the plurality of sensors: determining,by the central server, from the received second and third sensor data:prevailing speeds of the autonomous vehicles traversing the dedicatedroadway; vehicle dynamics of the autonomous vehicles traversing thededicated roadway; objects currently identified on the dedicatedroadway; and characteristics of the one or more trains traversing therailroad; and in response, determining, by the central server, one ormore actions for each of the autonomous vehicles for traversing thededicated roadway based on the prevailing speeds, the vehicle dynamics,the objects currently identified, and the characteristics of the one ormore trains traversing the railroad.

In some implementations, the method includes: acquiring, by theplurality of sensors, fourth sensor data of the autonomous vehiclestraversing the dedicated roadway; acquiring, by the plurality ofsensors, fifth sensor data indicative of a train that has derailed offthe railroad, the railroad traversing in parallel to the dedicatedroadway; transmitting, by the plurality of sensors, the acquired fourthand fifth sensor data to a central server; receiving, by the centralserver, the acquired second and third sensor data from each sensor ofthe plurality of sensors: determining, by the central server, from thereceived fourth and fifth sensor data: a first indication that the trainhas derailed off the railroad; a second indication that at least some ofthe autonomous vehicles traversing the dedicated roadway are on a pathto collide with the derailed train; and in response, transmitting, bythe central server, an instruction to the at least some of theautonomous vehicles to (i) reroute traffic on the dedicated roadway toavoid the derailed train, (ii) decelerate the autonomous vehicles, (iii)stop the autonomous vehicles from colliding with the derailed train, or(iv) a combination of (i)-(iii).

In some implementations, the method includes: acquiring, by theplurality of sensors, sixth sensor data of the autonomous vehiclestraversing the dedicated roadway of the roadway; detecting, by theplurality of sensors, an identity for each of the autonomous vehiclesfrom the sixth sensor data; determining, by the plurality of sensors, alocation for at least some of the autonomous vehicles on the dedicatedroadway; from the identity for each of the autonomous vehicles,determining, by the plurality of sensors, that the at least some of theautonomous vehicles are proximate to the end of the dedicated roadway;and in response, transmitting, by the plurality of sensors, anindication to the at least some of the autonomous vehicles to switch tothe normal processing mode.

In some implementations, the method includes wherein the normalprocessing mode comprises a setting for operating an autonomous vehiclewith an onboard trained machine-learning model used (i) prior toentrance of the autonomous vehicle to the dedicated roadway and (ii)after the autonomous vehicle exits the dedicated roadway.

The subject matter described in this specification can be implemented invarious embodiments and may result in one or more of the followingadvantages. Specifically, by augmenting the capabilities of the trainedmachine-learning model while the autonomous truck traverses the parallelroadway, the system described below can improve reliability and mitigatethe need for remote intervention for autonomous trucks. Similarly, thesystem can improve the safety of autonomous trucks operating theroadway. Within a railroad right of way environment, the system can,specifically, (i) can inform vehicles about various actors along theparallel roadway and (i) can inform vehicles about various trainstraveling in parallel on the railroad or over a shared roadway in theparallel roadway.

The details of one or more embodiments of the subject matter of thisspecification are set forth in the accompanying drawings and thedescription below. Other features, aspects, and advantages of thesubject matter will become apparent from the description, the drawings,and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a block diagram that illustrates an example of a system formonitoring autonomous vehicles traversing a dedicated roadway that runsalong railroad rights of way.

FIG. 1B is a block diagram that illustrates an example of a system fordetecting events on a dedicated roadway that runs along railroad rightsof way and notifying autonomous vehicles traversing the dedicatedroadway of the detected events.

FIG. 1C is another block diagram that illustrates an example of a systemfor monitoring autonomous vehicles traversing a dedicated roadway thatruns along railroad rights of way.

FIG. 2 is a block diagram that illustrates an example of components ofan autonomous vehicle using a normal operating mode and an enhancedoperating mode.

FIG. 3 is a flow diagram that illustrates an example of a process formonitoring autonomous vehicles traversing a dedicated roadway that runsalong railroad rights of way.

Like reference numbers and designations in the various drawings indicatelike elements.

DETAILED DESCRIPTION

FIG. 1A is a block diagram that illustrates an example of a system 100for monitoring autonomous vehicles traversing a dedicated roadway thatruns along railroad rights of way. The system 100, deployed upon aroadway 109 on which autonomous vehicles 108-1 through vehicle 108-N(collectively “vehicles 108”) travel, includes a plurality of sensors106-1 through 106-N (collectively “sensors 106”), a network 110, acentral server 112, a vehicle database 114, a railroad database 116, arailroad rights of way database 118, a train 102, and a railroad 104. Inthis example, the system 100 illustrates the processes performed by thesensors 106 and the central server 112. The system 100 illustrates twovehicles and eleven sensors, but there may be more or less sensors andmore or less vehicles, in other configurations. The roadway 109 is shownin system 100 with multiple lanes in a single direction. The roadway 109may alternatively or additionally include more or less lanes havingautonomous vehicles 108 travel in the same direction as well as morethan one lane of vehicles traveling in opposing directions. FIG. 1Aillustrates various operations in stages (A) through (I), which can beperformed in the sequence indicated, in another sequence, withadditional stages, or fewer stages.

In general, the system 100 can provide techniques for monitoringautonomous vehicles 108 on the roadway 109 and instructing autonomousvehicles 108 to take actions when the autonomous vehicles 108 enter adedicated road 109-1. The roadway 109 can include a dedicated road109-1. In some implementations, the dedicated road 109-1 can include oneor more lanes that run in parallel to a railroad 104. In someimplementations, the dedicated road 109-1 can include one or more lanesthat run in place of or over top of railroad 104. The dedicated road109-1 can be separate from the roadway 109 and can be accessed byegressing from the roadway 109.

In some implementations, the system 100 can be used in a drayageenvironment. In a drayage environment, goods can be transported bytrains and/or autonomous trucks over short distances. For example, thegoods can be transmitted from a ship that has entered at seaport to awarehouse, or from an inland port to a warehouse. The system 100 canutilize drayage in transferring shipments using various forms oftransportation.

The system 100 enables monitoring autonomous vehicles 108 traversing theroadway 109 and the dedicated road 109-1. In some examples, the vehicles108 can include autonomous vehicles or vehicles controlled by humans.The autonomous vehicles 108 can include and utilize one or more trainedmachine-learning models and an onboard sensor processing system.Functionally, the one or more trained machine-learning models canexecute in conjunction with the onboard sensor processing system toprovide navigation and driving capabilities for the autonomous vehicles108.

These autonomous vehicles 108 can obtain sensor data from its one ormore sensors that communicate with an onboard sensor processing systemand use the obtained sensor data to navigate the roadway 109. Forexample, autonomous vehicle 108-1 can analyze the obtained sensor databy providing the obtained sensor data as input to the one or moretrained machine-learning models. The one or more trainedmachine-learning models can output a likelihood detection of an event, aclassification of one or more objects illustrated in the sensor data,and other likelihoods of detected events. In response, the autonomousvehicles 108-1's route guidance system can analyze the output from theone or more trained machine-learning models to decide actions for theautonomous vehicle 108-1. These actions can include, for example, turnleft, turn right, accelerate, decelerate, stop, or reverse, to name afew examples.

However, the on-board capabilities of the autonomous vehicles 108 can beimpacted by external factors and may not function reliably. To improvethe capabilities of the autonomous vehicles 108, the system 100 candeliver supplemental processing to the onboard capabilities of theautonomous vehicles 108 during their traversal of the dedicated road109-1. More specifically, when the autonomous vehicles 108 traverse thededicated road 109-1 of the roadway 109, the system 100 can provide thesupplemental processing to the autonomous vehicles 108 to improvereliability and mitigating the operational burden of remotely monitoringand intervening in autonomous vehicle operations.

As will be further described in detail below, when autonomous vehiclesenter the dedicated road 109-1, the autonomous vehicles can receiveinstructions from sensors proximate to the dedicated road 109-1 toenhance its thinking. In this manner, the autonomous vehicles can switchto using an enhanced machine-learning model. The enhancedmachine-learning model can rely on not only sensor data generated bysensors onboard the autonomous vehicle but can also rely on sensor dataor instructions provided by the sensors proximate to the dedicated road109-1. The sensors monitoring the dedicated road 109-1 can offer insightdescribing events and detection of actors that may be unseen by theonboard sensors of the autonomous vehicles. As such, the enhancedmachine-learning model of the autonomous truck can have more visibilityof the dedicated road 109-1 using sensor data from both onboard sensorsand external sensors that monitor the dedicated road 109-1.

For example, the enhanced machine-learning model can receive inputs fromthe sensors that monitor the dedicated road 109-1. These inputs caninclude data indicating detected events on the dedicated road 109-1,actions for the autonomous vehicle to take while traversing thededicated road 109-1 based on the detected events, and other sensor dataas seen by the sensors monitoring the dedicated road 109-1. The enhancedmachine-learning model can also receive sensor data as input from itsown sensors onboard the autonomous vehicle and vehicle characteristicsof the autonomous vehicle. In this case, the autonomous vehicles caneffectively use both sets of sensor data for enriching the one or moretrained machine-learning models while traversing the dedicated road109-1 and utilize the actions produced by the enhanced machine-learningmodel to determine how to traverse the dedicated road 109-1.

The sensors 106 can include a variety of software and hardware devicesthat monitor objects on the roadway 109 and dedicated road 109-1. Forexample, the sensors 106 can include a LIDAR system, a video camera, aradar system, a BLUETOOTH™ system, weather components, and a Wi-Fisystem, to name a few examples. In some implementations, a sensor caninclude a combination of varying sensor types. For example, sensor 106-1can include a video camera and a radar system; sensor 106-N can includea video camera and a LIDAR system; and, sensor 106-3 can include a videocamera, a LIDAR system, and a Wi-Fi system. Other sensor combinationsare also possible.

A sensor can detect and track objects on the roadway 109 through itsfield of view. Each sensor can have a field of view set by the designerof system 100. For example, if sensor 106-7 includes a video camera, thefield of view of the video camera can be based on the type of lens used,e.g., wide angle, normal view, and telephoto, for example, and the depthof the camera field, e.g., 20 meters, 30 meters, and 60 meters, forexample. Other parameters for each sensor in system 100 can also bedesignated. For example, if the sensor 106-1 includes a LIDAR system,then the parameters required for its use would include a point density,e.g., a distribution of the point cloud, a field of view, e.g., angle inthe LIDAR system can view over, and line overlap, e.g., a measure to beapplied that affects ground coverage. Other parameters for each of thesensors are also possible.

The field of view of each sensor also becomes important because thesystem 100 can be designed in a variety of ways to enhance monitoring ofobjects on the roadway 109. For example, a designer may seek to overlapfields of view of adjacent sensors 106 to ensure continuity for viewingthe roadway 109 in its entirety. Overlapping field of view regions mayfacilitate monitoring areas where objects enter the roadway 109 throughvehicle on-ramps, exit the roadway 109 through vehicle off-ramps, ormerge onto different lanes. In some examples, the designer may decidenot to overlap the fields of view of adjacent sensors 106 but rather,juxtapose the fields of view of adjacent sensors 106 to ensure thewidest coverage of the roadway 109. In this manner, the system 100 canmonitor and track more vehicles on roadway 109 at a time.

In addition, each sensor can include memory and processing componentsfor monitoring the objects on the roadway 109. For example, each sensorcan include memory for storing data that identified and tracks theobjects in the order the vehicles appear to a sensor. Similarly, each ofthe sensors 106 can include processing components for processing sensordata, identifying the objects in the sensor data, generating the datathat identifies, and is later used to track the identified objects. Theprocessing components can include, for example, video processingcomponents, sensor-processing components, transmission components, andreceive components and/or capabilities. Each of the sensors 106 can alsocommunicate with one another over the network 110. The network 110 mayinclude a Wi-Fi network, a cellular network, a BLUETOOTH™ network, anEthernet network, or some other communicative medium.

The sensors 106 can also communicate with a central server 112 overnetwork 110. The central server 112 can include one or more serversconnected locally or over a network. The central server 112 can alsoconnect to one or more databases, e.g., a vehicle database 114, arailroad database 116, and right of way database 118. For example, thecentral server 112 can store data that represents the sensors 16 thatare available to be used for monitoring the roadway 109. The dataindicates which sensors 106 are active, which sensors 106 are inactive,the type of data recorded by each sensors, and data representing thefields of view of each sensors.

The central server 112 can store data identifying each of the sensors106 such as, for example, IP addresses, MAC addresses, and preferredforms of communication to each particular sensor. The data can alsoindicate the relative positions of the sensors 106 in relation to eachother. In this manner, a designer can access the data stored in thecentral server 112 to learn which sensors 106 are being used to monitorthe roadway 109, pertinent information for each of these sensors 106,and debugging information related to each of these sensors 106.

During stage (A), the sensors 106 deployed along roadway 109 cangenerate sensor data that represents autonomous vehicles 108 traversingthe roadway 109. The sensors 106 can be deployed longitudinally alongroadway 109, along both sides of the roadway 109, spaced a predetermineddistance apart from one another, and positioned so that its field ofview faces the roadway 109. Moreover, the sensors 106 can be configuredto generate sensor data of road actors, e.g., objects in the roadway109, autonomous vehicles 108 in the roadway 109, people walking inparallel to and perpendicular to roadway 109, and other objects.

The roadway 109 can include various types of roads. For example, thetypes of roads can include exit ramps, entry ramps, general-purposelanes, high occupancy vehicle (HOV) lanes, highways, back roads, sidestreets, and other roads. The other roads can include different types ofvarious capacity roads, larger roads, private roads, intersecting roads,and other thoroughfares that sensors 106 displaced along these roads cangenerator sensor data. The sensors 106 positioned along these roads cangenerate sensor data as they detect road actors entering their field ofview on the roadway 109. For example, the sensor data generated by eachof the sensors 106 can include an identification of a vehicle type,identification of an object type, characteristics of detected vehicles,vehicular congestion, vehicle dynamics, and vehicle density per unitarea, to name some examples.

The identification of the vehicle type can correspond to, for example, atruck, a sedan, a minivan, a hatchback, an SUV, and other vehicle types.The identification of the vehicle type can be based on a size of thevehicle, for example. Characteristics of the vehicle can include, forexample, vehicle color, vehicle size, wheelbase distance, length ofvehicle, height of vehicle, and width of vehicle. Vehicular density perunit area can correspond to a number of vehicles measured over aparticular area in traffic. Vehicular congestion can correspond to ameasure of an amount of traffic and movement rate of the traffic in aparticular area. Vehicle headway can correspond to a distance between afirst and second vehicle in a transit system measured in time or indistance. Vehicle dynamics can include acceleration, deceleration, andvelocity of one or more vehicles traveling along the prior roadways overa period of time.

In some implementations, the sensors 106 deployed at each of theseroadways can generate the sensor data at various intervals. For example,each time a sensor detects a vehicle in its field of view, the sensorcan generate the sensor data. In response to generating the sensor data,sensors 106-1 can transmit the generated sensor data to the next sensorin the longitudinal direction along the same roadway 109 to confirm thatit also detects similar sensor data. The next sensor can pass itsgenerated sensor data to the next sensor down the longitudinal line onthe roadway 109 to ensure it sees similar vehicles. In this manner, thegenerated sensor data is highly accurate because each sensor on theroadway 109 can confirm the prior sensor's generated sensor data. Insome examples, the sensors 106 can generate sensor data on a time basis,such as every 2 seconds. On the time basis, the sensors 106 may reducetheir bandwidth and processing, but ultimately include less accuratesensor data results.

For example, sensor 106-1 can detect that an autonomous vehicle 108-1has entered its field of view. In response to detecting, the sensor106-1 can record sensor data or media of a segment or portion of theroadway 109 and process the recorded sensor data using object detectionor some other form of classification to detect the moving object. Theobject detection can seek to identify a vehicle, a person, an animal, oran object on the roadway 109. The object may be stationary or moving. Inthe example of system 100, the sensor 106-1 can detect and classifyautonomous vehicle 108-1 on the main portion of roadway 109. Similarly,the sensors 106-1, 106-2, 106-3, and 106-8 will have processed vehicle108-N.

In some implementations, each of the sensors 106 can detect autonomousvehicle 108-1 by performing data aggregations of observations over awindow of time. The data aggregations can improve the sensors'detectability of a vehicle in its field of view. The data aggregationcan ensure that each sensor can identify and detect similar vehicles andtheir corresponding features.

The sensor 106-1 can then identify one or more features of theautonomous vehicle 108-1 detected in its field of view. These featurescan include observable properties of the vehicle, such as the vehiclecolor, e.g., as represented by red-green-blue (RGB) characteristics, thevehicle size, e.g., as calculated through optical characteristics, thevehicle class, e.g., as calculated through optical characteristics, andthe volume of the vehicle, as calculated through opticalcharacteristics. For example, the sensor 106-1 can determine thatautonomous vehicle 108-1 is a green colored vehicle, is over 110 ft³ insize, has a vehicle type of a sedan, and is a small sized vehicle. Thesensor 106-1 may also be able to determine one or more characteristicsof the vehicle, such as its rate of speed, the distance away from thesensor 106-1, the autonomous vehicle 108-1's direction of travel, and anumber of individuals found in the autonomous vehicle 108-1, to name afew examples.

In some implementations, the types of components found at the particularsensor that detect the vehicle can determine the characteristics thatdescribe the vehicle. For example, sensor 106-1 may include a videocamera and a radar system. The sensor 106-1 can then determinecharacteristics using the media recorded from the video camera and theelectromagnetic reflectivity from the radar system. For example, thesensor 106-1 can determine a color of the object, a size of the object,a distance from the object, a rate of movement of the object, and adirection of movement of the object. However, if the sensor 106-1 doesnot include the radar system, the sensor 106-1 can use other externalcomponents to determine the distance from the object, rate of movementof the object, and direction of movement of the object. For example, thesensor 106-1 may be able to utilize an external classifier to producethese results. The external classifier may be stored at the sensor 106-1or stored at a location accessible to the sensor 106-1 over network 110,e.g., such as the central server 112. Thus, the system 100 can benefitfrom having a combination of components to improve the detection processfound at each of the sensors.

In some implementations, the sensor 106-1 can generate other featuredata on the sensor data using sensor fusion. For example, in the casewhere sensor 106-1 utilizes multiple components, e.g., LIDAR, radar, anda video camera, the sensor 106-1 can combine the observation from eachof these components and assign these observations to a point in space.The point in space can correspond to an N-dimensional value thatdescribes the feature. Then, the sensor 106-1 can use features tocalculate and classify that particular point in space. For example, thesensor 106-1 can enjoin data from the LIDAR system, the radar system,and the video camera. The LIDAR system can generate 1 point percentimeter for 150-meter range for viewing the roadway 109, for example.The radar system can perform calculations that estimate where thevehicle or object is located in relation to the radar system. The videocamera can estimate a volumetric projection of the identified object orvehicle based on a volumetric projection estimation algorithm. Thesensor 106-1 can then calculate an identity product, e.g., the featuredata, using the observations from each of these sensors, which cancorrespond to a hash of the observations. For example, the sensor 106-1can calculate an identity product of the feature data and a timestampthe features were identified, from data provided by each of the sensors.

Then, the sensor 106-1 can transmit data representing the identityproduct of the feature data to the next sensor in the direction oftraffic, e.g., sensor 106-2. The sensor 106-1 may transmit the datarepresenting the identity product of the feature data when autonomousvehicle 108-1 has exited sensor 106-1's field of view. The datarepresenting the identity product of the feature data can include, forexample, a data structure, a matrix, or a link to data stored in adatabase. The sensor 106-1 can determine which sensor is the next sensorin a longitudinal line along the roadway 109. In some implementations,the sensor 106-1 may determine the next sensor by checking an order ofthe sensors. In some implementations, the sensor 106-1 may request fromthe central server 112 to indicate which sensor is the next sensor toreceive the data. In response to receiving an indication from thecentral server 112 indicating which sensor to transmit the data, e.g.,sensor 106-2, the sensor 106-1 can transmit the data representing theidentity product of the feature data to sensor 106-2 over network 110.

The sensor 106-2 can receive the identity product of feature data fromthe sensor 106-1. The sensor 106-2 can generate feature data when itdetects autonomous vehicle 108-1 in its field of view. In response togenerating the feature data, the sensor 106-2 can compare the generatedfeature data with the received feature data from sensor 106-1. If thecomparison results in a match or a near match within a threshold value,then the sensor 106-2 can determine that it is viewing the sameautonomous vehicle 108-1 as seen by sensor 106-1. In some examples,sensor 106-2 may transmit a confirmation back to sensor 106-1 indicatingthat it saw the same vehicle. Then, when autonomous vehicle 108-1 exitsthe field of view of sensor 106-2, the sensor 106-2 can transmit thegenerated feature data to the next sensor down the roadway 109, e.g.,sensor 106-3. Each sensor within system 100, e.g., sensors 106-1 through106-N, can perform a similar process when a vehicle is detected in itsfield of view.

In some implementations, the sensors can transmit their respectivesensor data to the central server 112 each time a new object isdetected. In some examples, the sensors can transmit their respectivesensor data when a sensor receives confirmation from the next sensordown the longitudinal line of sensors. The generated sensor data can notonly include data regarding detected objects, but data identifying thesensors. The data identifying the sensors can include, for example, atype of sensor, the data generated by the sensor, IP addresses of thesensor, and MAC addresses of the sensor.

The central server 112 can receive the sensor data from each of thesensors. In some examples, the central server can access one or moredatabases to retrieve the generated sensor data from each of thesensors. In response, the central server 112 can generate vehicularcharacteristics from the generated sensor data. The vehicularcharacteristics can include, for example, prevailing speeds of thevehicles, vehicle dynamics, sensor visibility, object identification,and train characteristics.

For example, the prevailing speeds of the vehicles along the roadway 109can correspond to the speed at which 85 percent of the vehicles aretraveling at or below that speed. The central server 112 can use thecalculated prevailing speed as a reference for the speeds at which theautonomous vehicles 108 should travel along the dedicated road 109-1.The central server 112 can determine vehicle dynamics of autonomousvehicles 108 currently traversing the roadway 109. The vehicle dynamicscan include vehicle acceleration, vehicle speed, and vehicledeceleration. Moreover, the central server 112 can determine sensorvisibility, and determine whether the sensors can accurately see theroad actors on the dedicated road 109-1. The central server 112 candetermine from the sensor visibility whether a sensor is too close toanother sensor, as the sensors share overlapping fields of view, andwhether the sensors are too close or too far from the roadway 109. Inresponse to generating this information, the central server 112 can aidthe sensors monitoring the roadway 109 in determining actions for thevehicles to take. For example, based on current detected speeds ofvehicles and identification of trains in the sensors data, the centralserver 112 can instruct the sensors to instruct the autonomous vehiclestraversing the dedicated road 109-1 to take a specific action, e.g.,slow down, accelerate, or stop, to name a few examples.

Similarly, the autonomous vehicle 108-1 may include one or more sensors,an onboard processing sensor system, and one or more trainedmachine-learning models. As autonomous vehicle 108-1 traverses theroadway 109, the sensors of autonomous vehicle 108-1 can obtain sensordata in a continuous fashion. The sensor data can include, for example,video, audio, LIDAR data, radar data, and other data types. The sensordata can illustrate an environment proximate to the autonomous vehicle108-1 as seen by its sensors. The environment can include, for example,a portion of the roadway 109, traffic signs, traffic lights, mergelanes, transition lanes, exit lanes, continuous lanes, objects in theroadway 109, the railroad 104, train 102, and other data. The sensors ofautonomous vehicle 108-1 (and the other autonomous vehicles) can obtainsensor data in a continuous or periodic fashion, to name a few examples.

In some implementations, the onboard sensor system can obtain currentvehicle characteristics. Specifically, the onboard sensor system cancommunicate with various devices in the autonomous vehicle 108-1's usingthe controller area network (CANBUS) system. The CANBUS system canprovide a means for the onboard sensor system to obtain informationrelated to the autonomous vehicle 108-1's characteristics. Thesecharacteristics can include, for example, data related to autonomousdriving, advance driver assistance system (ADAS), transmission, airbags,antilock braking (ABS), cruise control, electric power steering, audiosystems, power windows, doors, mirror adjustment, battery and rechargingsystems, and vehicle dynamics, e.g., vehicle speed. For example, theonboard sensor system can communicate with the engine control unit (ECU)using the CANBUS system to obtain vehicle characteristic information.

During stage (B), the onboard sensor system can provide the sensor dataand the vehicle characteristics as input to the one or more trainedmachine-learning models. For example, the one or more trainedmachine-learning models can receive as input video, audio, images, LIDARdata, radar information, current vehicle characteristics information,and other data types. These data types can be in the form of imagefiles, binary files, and other file types. The one or more trainedmachine-learning models can process the received inputs through each ofthe nodes in the models. The one or more trained machine-learning modelscan receive inputs and generate outputs on a continuous basis or eachtime the sensors obtain new input data.

During stage (C), the one or more trained machine-learning models canoutput a likelihood detection of an event, a classification of one ormore objects illustrated in the sensor data, and other detected eventsin response to processing the inputs. For example, as illustrated insystem 100, the one or more trained machine-learning models can output adetection of 99% of an obstacle free zone on roadway 109. This outputcan indicate to the route guidance system of the autonomous vehicle108-1 that the portion of roadway 109 as seen by onboard sensors doesnot detect an obstacle, an object, or other blocking device on roadway109 with 99% confidence.

The one or more trained machine-learning models can also output otherdetection types and confidence levels. For example, the one or moretrained machine-learning models can output a 70% detection of a deer onroadway 109, a 90% detection of a train on roadway 109, e.g., indicativeof train 102 that has fallen off the railroad 104 and onto the roadway109, a 30% detection of rainy or ice on roadway 109, and other detectiontypes. The one or more trained machine-learning models can output alikelihood of an event and a description of an event depicted in theinput. In response to generating the output, the onboard sensorprocessing system can provide the output to a route guidance system ofthe autonomous vehicle 108-1.

During stage (D), the route guidance system of the autonomous vehicle108-1 can receive the output from the one or more trainedmachine-learning models. The route guidance system can include one ormore algorithmic processes that can monitor a location of a vehicle inreal time, e.g., via geographic coordinate system (GPS), and map thelocation of the vehicle on a digital map. For an autonomous vehicle, theroute guidance system can ensure the autonomous vehicle 108-1 follows aroute guidance from an origin location to a destination location.

The route guidance system can identify a path for the autonomous vehicle108-1 to travel from an origin location to a destination and ensure theautonomous vehicle 108-1 reaches the destination safely. Specifically,the route guidance system can produce actions for the vehicle to takewhile traversing to the destination. These actions can include, forexample, accelerate, change lanes, stop, decelerate, turn left, turnright, U-turn, and other actions. The route guidance system can rely onoutputs from the one or more trained machine-learning model to produceactions for the autonomous vehicle to take while traversing to thedestination. For example, if the route guidance system determines thatthe one or more trained machine-learning models determines a 99%likelihood of obstacle free zone, then the route guidance system candetermine that the autonomous vehicle 108-1 continues on its guided pathto the destination.

Alternatively, if the one or more trained machine-learning modelsdetermine a 99% likelihood of an identified obstacle on the roadway 109,and then the route guidance system can determine an action for theautonomous vehicle 108-1 to avoid the obstacle. These actions to avoidthe obstacle can include, for example, stopping until the obstacle hascleared, slowing down to let the obstacle pass off the roadway 109,changing lanes to avoid the obstacle, and other actions. The routeguidance system can continuously output actions for the autonomousvehicle 108-1 to take based on a monitoring of the route guidance pathand the output provided by the one or more trained machine-learningmodels.

In some implementations, an external party may set a route guidance pathfor the autonomous vehicle 108-1 to travel. The external party, whichmay include a human or a computer system, may set the route guidancepath for the autonomous vehicle 108-1 to travel before the autonomousvehicle 108-1 departs for the destination. Similarly, the autonomousvehicle 108-1 may receive a route guidance path while in transit to adestination and may receive updates to the route guidance path while intransit to the destination. In some examples, the route guidance pathcan include, for example, a GPS location of a destination, a path forthe route guidance system to follow from an origin to a destination, aname of a destination, and other data specifying the origin location,the destination, and the path for the route guidance system to follow.

During stage (E), the route guidance system of the autonomous vehicle108-1 can produce an action to take. As illustrated in system 100, forexample, the action can include “Turn Left.” As depicted by the dottedline in system 100, the autonomous vehicle 108-1 can turn left from theroadway 109 to the dedicated road 109-1. As previously mentioned, thededicated road 109-1 can include one or more lanes that run in parallelto the railroad 104. In response to producing an action, the routeguidance system can instruct the autonomous vehicle 108-1 to move inaccordance with the action. For example, if the route guidance systeminstructs the autonomous vehicle 108-1 to turn left, then the routeguidance system can instruct various components of the vehicle, e.g.,steering wheel, axel, tires, accelerator, brake, etc., to collectivelymove the autonomous vehicle 108-1 to make a left turn. Similarly, theroute guidance system can instruct the autonomous vehicle 108-1 to takeother actions as well.

In some implementations, the route guidance system of the autonomousvehicle 108-1 can receive an instruction from an external party toinstruct the autonomous vehicle 108-1 to enter the dedicated road 109-1.In some implementations, the route guidance system of the autonomousvehicle 108-1 can automatically generate an instruction that instructsthe autonomous vehicle 108-1 to enter the dedicated road 109-1. Theseinstructions can come from a prior route guidance party or during theautonomous vehicle 108-1's current traversal on roadway 109.

In some implementations, the sensors 106 can monitor the path oftraversal of the autonomous vehicle 108-1 on the roadway 109. Forexample, as the autonomous vehicle 108-1 enters and subsequently exitsthe fields of view of sensors 106-1, 106-2, and 106-3, these specificsensors can identify the autonomous vehicle 108-1 and detect itsmovement. However, after sensor 106-3 detects the autonomous vehicle108-1 entering and exiting its field of view, the sensor 106-3 cantransmit its identity product of feature data to both sensors 106-4 andsensor 106-8. The sensors can transmit the identity product of featuredata to multiple sensors when the roadway 109 splits in differentdirections. By transmitting the identity product of feature data tomultiple sensors, e.g., sensor 106-4 and sensor 106-8, the sensors 106can continuously monitor the path of autonomous vehicle 108-1's movementwhen the roadway 109 travels in different directions.

For example, if the sensor 106-8 determines a vehicle entered its fieldof view and determines that the identity product of feature datareceived from sensor 106-3 matches to the feature data generated bysensor 106-8, then sensor 106-8 can determine that autonomous vehicle108-1 is the same vehicle seen by sensor 106-3-8, and that theautonomous vehicle 108-1 is traversing down roadway 109. Alternatively,if the sensor 106-4 determines a vehicle entered its field of view anddetermines that the identity product of feature data received fromsensor 106-3 matches to the feature data generated by sensor 106-4, thensensor 106-4 can determine that autonomous vehicle 108-1 is the samevehicle seen by sensor 106-3, and that the autonomous vehicle 108-1 hasturned into the dedicated road 109-1 from the roadway 109. Asillustrated in system 100, autonomous vehicle 108-1 has departed theroadway 109 and entered the dedicated road 109-1.

In response to sensor 106-4 detecting that it has seen the same vehiclein its field of view as a previous, subsequent sensor, e.g., sensor106-3, then sensor 106-4 can transmit the identity product of featuredata to each of the other sensors. In this manner, the other sensors 106can also seek to determine whether they see one or more similar vehiclesin their field of view. This ensures the sensors can track each of thevehicles 108 as they traverse the roadway 109 and the dedicated road109-1. In some implementations, the sensor 106-4 can transmit anotification to the autonomous vehicle 108-1 upon entering the dedicatedroad 109-1 to switch to an enhanced machine-learning model in responseto detecting the autonomous vehicle 108-1's entry to the dedicated road109-1.

During stage (F), the autonomous vehicle 108-1 can detect entry into thededicated road 109-1. In some implementations, the entry into thededicated road 109-1 can include a toll. The toll can charge a customeror owner of the autonomous vehicle 108-1 upon passing through the toll.The toll can include, for example, a radio frequency ID reader, tollplazas, tollbooths, tollhouses, toll stations, toll bars, toll barriers,or tollgates, to name a few examples. Some tolls can be automaticallycharged and some tolls may be manually charged. In the case ofautonomous vehicles, tolls can charge the autonomous vehicles withelectronic toll collection equipment which can automatically communicatewith the autonomous vehicle's transponder or use automatic vehicle platerecognition to charge the vehicles by debiting corresponding accounts.The charged toll can be used to generate revenue operator withoutmaterially adversely impacting the existing rail business. In someexamples, the charged toll may be at a higher operating margin than whatthe railroad operator typically charges for railroad operation. In someexamples, the charged toll may cost a similar amount to what therailroad operator typically charges for railroad operation.

In some implementations, a marker can signal the entry into thededicated road 109-1. The marker can include, for example, a line on thededicated road 109-1, a sign indicating “Entry into Railroad ROW,” audioindicating entry into the dedicated road 109-1, a speed bump, and otherindicators. The sensors onboard the autonomous vehicle 108-1 can detectthe marker, and signify to the route guidance system of its entry intothe dedicated road 109-1. In some implementations, the onboard sensorsystem of the autonomous vehicle 108-1 can switch to an enhancedmachine-learning model in response to detecting the autonomous vehicle108-1's entry to the dedicated road 109-1 using the marker. In someimplementations, the onboard sensor system of the autonomous vehicle108-1 can switch to an enhanced machine-learning model in response toreceiving a notification from the sensors monitoring the dedicated road109-1 to switch its processing capabilities to the enhanced mode.

During stage (G), the onboard sensor system of the autonomous vehicle108-1 can set the one or more trained machine-learning models asenhanced in response to detecting its entry into the dedicated road109-1. For example, in response to detecting entry into the dedicatedroad 109-1, the onboard sensor system of the autonomous vehicle 108-1can perform at least one of the following functions: (i) enabling theone or more trained machine-learning models to receive additional inputsrelated to the sensor data from sensors monitoring the dedicated road109-1, (ii) deleting the one or more trained machine-learning modelsfrom memory to access different enhanced machine-learning models, (iii)removing the one or more trained machine-learning models from cachememory and storing the one or more trained machine-learning models inmain memory, and (iv) transmitting the one or more trainedmachine-learning models to the central server 112 for later retrievaland removing the one or more trained machine-learning models frommemory, to name some examples. Generally, accessing and instantiatingthe one or more enhanced trained machine-learning models enables theautonomous vehicle 108-1 to be better prepared for events occurring onthe dedicated road 109-1. Specifically, by activating the enhancedtrained machine-learning models on the autonomous vehicle 108-1, theonboard sensor system, and the route guidance system can determineactions that are safer and more reliable during the autonomous vehicle108-1's entire traversal of the dedicated road 109-1.

During stage (H), in response to setting the one or more trainedmachine-learning models as enhanced, the onboard sensor system of theautonomous vehicle 108-1 can activate an enhanced machine-learning modelfor further processing. In some examples, the onboard sensor system caninsert the enhanced machine-learning model in cache memory to enableaccessing the enhanced machine-learning model on a more frequent basis.In some examples, the onboard sensor system can request the enhancedmachine-learning model from the central server 112. In this example, theonboard sensor system can transmit a request to the central server 112over network 110 for the enhanced machine-learning model andsubsequently receive the enhanced machine-learning model in responsefrom the central server 112.

During stage (I), the onboard sensor system of the autonomous vehicle108-1 can activate one or more sensors for communication purposes. Insome implementations, the onboard sensor system of the autonomousvehicle 108-1 can activate one or more sensors for communication inresponse to activating the enhanced machine-learning model. The one ormore sensors for communication purposes can include, for example, Wi-Ficapabilities, cellular capabilities, BLUETOOTH™ capabilities, and othernetwork communication capabilities. The onboard processing system of theautonomous vehicle 108-1 activates the one or more communication sensorswhile the autonomous vehicle 108-1 traverses the dedicated road 109-1because the sensors monitoring the dedicated road 109-1 can communicatedata to the autonomous vehicle 108-1 for navigation.

Specifically, as the autonomous vehicle 108-1 traverses the dedicatedroad 109-1, the autonomous vehicle 108-1 can utilize the data providedby these sensors to support the decision making for the autonomousvehicle 108-1. In some implementations, the onboard sensor system canreceive sensor data from the sensors monitoring the dedicated road 109-1and provide the received sensor data to the enhanced machine-learningmodel to produce an enhanced output. Moreover, the enhancedmachine-learning model can also receive sensor data from the sensorsonboard the autonomous vehicle 108-1 and data indicative of the vehiclecharacteristics to augment the decision-making capabilities of theenhanced machine-learning model. The enhanced output can indicate alikelihood of a detected event or a likely action for the autonomousvehicle 108-1 to take while traversing the dedicated road 109-1. Inresponse, the onboard sensor system can provide the enhanced output tothe route guidance system for generating one or more actions for theautonomous vehicle 108-1 to take. The various actions and decisions thatthe autonomous vehicle 108-1 can take while traversing the dedicatedroad 109-1 will be further described below.

As illustrated in system 100, sensors 106-4 through 106-7, andsubsequently sensors 106-11 through 106-17 shown in FIGS. 1B and 1C,respectively, can monitor passage of vehicles through the dedicated road109-1. In some implementations, one or more sensors can monitor anentryway of the dedicated road 109-1. Specifically, one or more sensorsproximate to entry of the dedicated road 109-1 can be configured tomonitor the entryway of the dedicated road 109-1. For example, sensors106-4 and 106-5 can be configured to monitor the areas that include andare proximate to the marker at the entry of the dedicated road 109-1. Inthis example, sensors 106-4 and 106-5 can have their fields of viewcover areas within and proximate to the marker at the entry of thededicated road 109-1.

In some implementations, when sensor 106-4 (i) receives an identityproduct of feature data from sensor 106-3 and (ii) determines thevehicle seen in its field of view matches to the vehicle seen by sensor106-3, the sensor 106-4 can be configured to take additional actions. Insome implementations, when sensor 106-4 or sensor 106-5 detects anobject in its field of view at the entry of the dedicated road 109-1,the sensors 106-4 or 106-5 can be configured to take the additionalactions. The latter implementation can be performed without sensor 106-3notifying sensors 106-4 and 106-5 of a detected vehicle. The additionalactions can include, for example, transmitting a notification to thedetected vehicle to switch to an enhanced processing mode, notifyingother sensors monitoring the dedicated road 109-1 of the detection of avehicle entering the dedicated lane, transmitting a notification to thecentral server 112 indicating a vehicle has entered the dedicated lane,a combination of the above actions, or a different action.

For example, as illustrated in system 100, sensor 106-4 can detectautonomous vehicle 108-1 approaching the marker at the entryway of thededicated road 109-1 and subsequently entering the dedicated road 109-1.In response to detecting the autonomous vehicle 108-1 entering thededicated road 109-1, the sensor 106-4 can transmit a notification tothe autonomous vehicle 108-1 to switch to an enhanced processing mode.In some implementations, the autonomous vehicle 108-1 may have switchedto the enhanced processing mode prior to receiving the notification fromone or more sensors monitoring the dedicated road 109-1 or the entrywayof the dedicated road 109-1. In this implementation, the onboardprocessing system of autonomous vehicle 108-1 can receive thenotification from the sensor 106-4, for example, and in response cantransmit a notification to the sensor 106-4 confirming the switch to theenhanced processing mode has been performed.

In response, the sensor 106-4 can transmit a confirmation to each of thesensors monitoring the dedicated road 109-1 indicating that the vehicletraversing the dedicated lane has switched to the enhanced processingmode. In this manner, each of the sensors, e.g., sensors 106-4 through106-7 and 106-11 through 106-17, can ensure that the autonomous vehicle108-1 is prepared to receive instructions from these sensors. If thesensor 106-4 transmits a notification to the autonomous vehicle 108-1and does not receive a confirmation back within a predetermined periodof time, then the sensor 106-4 can transmit a notification to thecentral server 112 indicating that the autonomous vehicle that hasentered the dedicated road 109-1 is not properly communicating. Thecentral server 112 can receive this notification and notify theauthorities that a vehicle traversing the dedicated lane may be an atrisk vehicle and should be inspected by the authorities for unsafedriving. In this case, the sensors 106-4 through 106-17 can continue tosend instructions to the autonomous vehicle 108-1 to cease driving thededicated road 109-1 until the autonomous vehicle 108-1 returns aconfirmation message indicating a switch to the enhanced processing modehas been performed.

In some implementations, the sensors 106 monitoring the dedicated road109-1 can also monitor the railroad 104. Specifically, the sensors 106-4through 106-17 can monitor train activities on railroad 104. Thesesensors 106-4 through 106-17 may include, for example, omni-directionalcapability that enables these sensors to obtain sensor data from eachdirection simultaneously, in a 360-degree fashion. In this manner, thesensors 106-4 through 106-17 can not only monitor autonomous vehicles108 entering and traversing the dedicated road 109-1 but also one ormore trains, e.g., train 102, traversing the railroad 104. Should anautonomous vehicle accidentally cross onto the railroad 104, then thesensors 106 can notify the railroad system of a vehicle on the railroad104. Alternatively, should the train 102 fall off the railroad 104 ontothe dedicated road 109-1, the sensors 106 can notify the one or moreautonomous vehicles traversing the dedicated road 109-1 of actions totake to avoid the fallen train 102.

In some implementations, the railroad 104 and the dedicated road 109-1may overlap with one another. For example, a center of the dedicatedroad 109-1 may include the railroad 104. In this case, the autonomousvehicles 108 can traverse the combined roadway when the train 102 is notsimultaneously traversing the combined roadway, and the train 102 cantraverse the combined roadway when the autonomous vehicles 108 are notsimultaneously traversing the combined roadway. The sensors 106monitoring the combined roadway can send a notification to autonomousvehicles 108 seeking to enter the dedicated road 109-1 upon detection towait before entering the dedicated road 109-1 should a train 102 betraversing the dedicated road 109-1. Once the train 102 has passedthrough the combined roadway, the sensors 106 can transmit anotification to the autonomous vehicles 108 seeking to enter thededicated road 109-1 signaling it is safe to enter the dedicated road109-1. Here, the sensors 106 monitoring the dedicated road 109-1 thatincludes the railroad 104 can generate sensor data of both detectedautonomous vehicles and the train 102 and provide actions for thedetected autonomous vehicles to take based on the generated sensor data.

In some implementations, the sensors 106 can determine that the train102 has priority over autonomous vehicles 108 for traversing thecombined roadway. In some examples, the sensors 106 can set the train102 as having priority over the autonomous vehicles 108 because thetrain 102 cannot receive communications from the sensors 106. In someexamples, the sensors 106 can set the train 102 as having priority overthe autonomous vehicles 108 based on instructions provided from therailroad system or the central server 112.

In some implementations, the central server 112 can store one or moredata components of system 100. Specifically, the central server 112 canstore the one or more trained machine-learning models from each of thevehicles that traverse the roadway 109 and the dedicated road 109-1. Thecentral server 112 can store the enhanced machine-learning model used byeach of the autonomous vehicles 108 that traverse the dedicated road109-1. The central server 112 can receive requests from one or morevehicles for retrieval of the enhanced machine-learning-model, forstoring the one or more trained machine-learning models associated withan autonomous vehicle while the autonomous vehicle traverses thededicated road 109-1, and for retrieving the one or more trainedmachine-learning models associated with an autonomous vehicle after theautonomous vehicle exits the dedicated road 109-1.

The central server 112 can store each of the abovementioned datacomponents to alleviate the memory constraint required by each of theautonomous vehicles for storing the machine-learning models. In thismanner, when the one or more trained machine-learning models are not inuse on the autonomous vehicles, they can be stored on the central server112. When needed, the autonomous vehicles can transmit requests for aspecific model or set of models from the central server 112. In someimplementations, the central server 112 can store the data related toeach of the models in a vehicle database 114.

The vehicle database 114 can store data indicating one or more vehiclesthat traverse the dedicated road 109-1. Specifically, the vehicledatabase 114 can store indexing information that identifies a vehicleand associates the index information with data related to the vehicle.For example, the indexing information can include an IP address, a MACaddress, or another address related to the device that communicated amessage from the onboard sensor system of the autonomous vehicle, e.g.,autonomous vehicle 108-1.

The data related to the vehicle can include, for example, the enhancedmachine-learning model, one or more trained machine-learning models usedby the vehicle, and historic data related to the vehicle. The enhancedmachine-learning model may be specific to a particular autonomousvehicle. Similarly, the one or more trained machine-learning models maybe specific to a particular autonomous vehicle. As such, the centralserver 112 can track, store, train, and update the variousmachine-learning models according to specific vehicle configurations.The historic data can include, for example, a number of times thecorresponding vehicle has accessed the dedicated lane, a number of timesthe corresponding vehicle has been detected by the sensors 106, and anumber of times the corresponding vehicle has been reported by thesensors 106 as not having confirmed receipt of performing the switch tothe enhanced processing mode, to name a few examples. Other examples arealso possible.

In some implementations, the central server 112 can also receiverequests from one or more of the sensors for notifying the authorities.One or more of the sensors 106 can detect a vehicle that is drivingunsafely on the dedicated road 109-1 or failing to comply with thesensor provided instructions. In response, the one or more of thesensors 106 can transmit a notification to the central server 112indicating a detected vehicle is driving unsafely on dedicated road109-1, for example. The central server 112 can receive the notificationand notify the proper authorities in response to try and prevent anyfurther accidents or damage on the dedicated road 109-1. In thisexample, the one or more sensors can provide sensor data illustratingthe corresponding vehicle, the identity product of the feature data asdetermined by the sensor data of the vehicle, and other data thatrepresents the vehicle traversing unsafely on the dedicated road 109-1.

In some implementations, the central server 112 can also communicatewith a railroad database 116 and a right of way (ROW) database 118. Therailroad database 116 can include data related to the activities oftrain 102. For example, the activities can include a number of tripstaken by the train 102 on railroad 104, actual start times for eachtrip, actual end times for each trip, planned start times for each trip,planned end times for each trip, future planned trips for the train 102on railroad 104, profit received for operating the train 102 on railroad104, contact information for an operator of the train 102, and dataidentifying a railroad system that manages the train 102 and therailroad 104, to name a few examples.

The data identifying the railroad system can include data identifying aninterface that receives data from an external user or external systemfor managing the train 102 and the railroad 104. A client device, acomputing device, or another device can provide the interface. Anindividual, such as a train manager, can provide data indicative of thetrain 102 and the railroad 104 to the interface. In some example, therailroad system can be a computer system that can provide dataindicative of the train 102, data indicative of the railroad 104, dataindicative of past trips taken by trains on the railroad 104, and dataindicative of future trips on the railroad 104. Subsequently, thecentral server 112, one or more other devices in system 100, the sensors106, and the autonomous vehicles 108 can access data provided throughthe interface in system 100.

The data indicative of the train 102 and the railroad 104 that can bereceived by the interface and subsequently transmitted to variousdevices in system 100 can include, for example, a number of carsconnected on train 102, a time for an upcoming trip of train 102, anymechanical issues or failures related to train 102, contact informationfor a train operator, or dynamic characteristics related to the train102, e.g., train speed, acceleration, and direction of travel, to name afew examples.

Similarly, the devices of system 100 can transmit requests to theinterface requesting for information. For example, the sensors 106, theautonomous vehicles 108, and the central server 112 can transmit arequest to the interface for information related to the train 102 andrailroad 104. The request can include, for example, a predicted timewhen the train 102 is to reach a destination, e.g., a location proximateto the dedicated road 109-1 or to an end destination, a current locationof train 102, and additional status information related to train 102.The sensors 106, the autonomous vehicles 108, and the central server 112can receive responses from the interfaces. The responses can includeinformation pertinent to the request. For example, the sensors 106 canuse the train information provided from the interface to makedeterminations about instructions to provide to one or more autonomousvehicles 108-1 traversing the dedicated road 109-1. This will be furtherdescribed in detail below.

In some implementations, the ROW database 118 can store informationrelated to the dedicated road 109-1. This information can include, forexample, data identifying sensors that monitor the dedicated road 109-1,data identifying inactive sensors and active sensors that are positionedto monitor the dedicated road 109-1, and data identifyingcharacteristics of the dedicated road 109-1. The data identifying thesensors monitoring the dedicated road 109-1 can include, for example, IPaddresses, MAC addresses, and hostnames, as well as, the type of sensorsincluded in each of the sensors 106. For example, sensor 106-4 caninclude a LIDAR system and a video camera. The data identifying inactiveand active sensors can be, for example, a notification indicatingsensors 106-4, 106-5, 106-7, and 106-11 through 106-17 as active.Similarly, this data can indicate that sensor 106-6 is inactive.

The data identifying characteristics of the dedicated road 109-1 caninclude, a number of lanes in the dedicated road 109-1, a length of thededicated road 109-1, a direction of travel for each lane, a frequencyof use for the dedicated road 109-1, a location of the marker, and datarelated to the toll charged amount for using the dedicated road 109-1.The data related to the toll charged amount can include, for example, atotal amount of toll charged, a total amount of tolls received from theautonomous vehicles, a total amount of tolls not received from theautonomous vehicles, data identifying the transponders of the autonomousvehicles, and contact information related to the owner of the autonomousvehicles.

The central server 112 can use the information stored in the ROWdatabase 118 to charge users that own the autonomous vehicles that driveon the dedicated road 109-1 and do not pay upon entry. Specifically, thecentral server 112 can transmit a request for pay to the contactinformation of the owner for the charged toll amount plus a fee for notpaying the toll upon entry of the dedicated road 109-1. The centralserver 112 can receive the payment amount from the owner in response totransmitting the request to the owner, e.g., via cash, a check, apayment application, and payment through a website, to name someexamples. Similarly, the central server 112 can obtain paymentinformation related to railroad 104 usage. The payment information caninclude an amount the railroad management system charges for a train 102to use the railroad 104.

As such, the central server 112 can determine financial amounts relatedto tolls charged to vehicles and financial amounts related to trainstraversal of railroad 104. The central server 112 can produce analyticsthat describe, for example, profits related to using both the dedicatedroad 109-1 and the railroad 104, profits related to the individual usageof the dedicated road 109-1 and the railroad 104, and profit marginsrelated to the usage of the dedicated road 109-1 and the railroad 104.Other examples are also possible.

FIG. 1B is a block diagram that illustrates an example of a system 101for detecting events on a dedicated roadway that runs along railroadrights of way and notifying autonomous vehicles traversing the dedicatedroadway of the detected events. The system 101 is a continuation ofsystem 100. Thus, the functions described with respect to system 101 canalso be performed in system 100. Specifically, the system 101illustrates the autonomous vehicle 108-1 traversing the dedicated road109-1, which runs in parallel to the railroad 104. Moreover, the system101 illustrates sensors 106-11 through 106-14 that monitor the vehicles'traversal along the dedicated road 109-1. The monitoring can include,for example, detecting events on the dedicated road 109-1, notifyingvehicles traversing the dedicated road 109-1 of the detected events, andproviding instructions to the vehicles of actions to take based on thedetected events. FIG. 1B illustrates various operations in stages (J)through (M), which can be performed in the sequence indicated, inanother sequence, with additional stages, or fewer stages. The stages(J) through (M) follow the stages of (A) through (I) of FIG. 1A.

During stage (J), the sensors 106-11 through 106-14 can monitor thededicated road 109-1. In some implementations, the sensors 106-11through 106-14 can monitor the dedicated road 109-1 and the railroad104. The sensors 106-11 through 106-14 may include omni-directionalcapabilities. Similarly, the railroad 104 and the dedicated road 109-1may overlap with one another. Specifically, the sensors 106-11 through106-14 can generate sensor data on a frame-by-frame basis. The sensordata can include image data, video data, LIDAR data, radar data, anddata recorded from other sensor types, to name a few examples.

The sensors can process the sensor data to identify events detected inthe sensor data. In some examples, a sensor may detect in a frame ofLIDAR data an animal crossing the dedicated road 109-1. In someexamples, a sensor may detect in a frame of video data a 4×4 autonomoustruck traversing the dedicated road 109-1. In some examples, a sensormay detect in a frame of video data ice on the dedicated road 109-1.Other examples are also possible.

As illustrated in system 101, sensor 106-14 can obtain sensor data. Thesensor data can be obtained from sensor 106-14's field of view thatmonitors both the railroad 104 and the dedicated road 109-1. The sensordata can illustrate, for example, a tree fallen across the dedicatedroad 109-1. In response to detecting the event on the dedicated road109-1, the sensor 106-14 can notify other sensors that monitor thededicated road 109-1. Specifically, the sensor 106-14 can transmit anotification to the other sensors that includes, for example, dataindicating the detected event, the generated sensor data, the generatedidentity product, a timestamp associated with the sensor 106-14'sdetection of the event, and data indicating a significance level ofevent.

The significance level of event can be determined based on how impactfula detected event is to the autonomous vehicles 108 traversing thededicated road 109-1 or the train 102 traversing the railroad 104. Insome examples, if the event is determined to block the flow of trafficon the dedicated road 109-1 or block the railroad 104, e.g., a treefalling on the dedicated road 109-1 or the railroad 104, then the sensor106-14 can determine a high significance of the event. In some examples,if the event is determined to not block the flow of traffic on thededicated road 109-1 or on the railroad 104 or block the flow of trafficonly momentary, then the sensor 106-14 can determine a low significanceof the event. In some examples, the significance level of the event canbe based on a potential amount of money lost during a timeframe of thedetected event. In this example, the sensor 106-14 can determine thesignificance level of the detected event is high because the treeblocking the dedicated road 109-1 ceases the flow of traffic, which,ceases the flow of tolls being charged, and ultimately reduces theamount of profit for the system 101. Other examples are also possible.

During stage (K), the sensor 106-13 can receive the notification fromthe sensor 106-14 over the network 110. In response to receiving thenotification, the sensor 106-13 can process the notification anddetermine that the transmitting sensor, e.g., sensor 106-14, identifiedan event and determined the significance level of the event. Forexample, the sensor 106-13 can determine from the notification anidentification of a fallen tree in the dedicated road 109-1 and thesignificance level of the event to be high. The sensor 106-13 cangenerate sensor data from its field of view to determine whether it alsodetects the fallen tree or another object from the sensor data. If thesensor 106-13 does not detect the fallen tree or another object, thenthe sensor 106-13 can transmit (i) the notification received from sensor106-14 and (ii) a notification that includes, for example dataindicating no event was detected, the generated sensor data, thegenerated identity product, a timestamp associated with the sensor106-13's generated sensor data, and data indicating no significance. Bytransmitting the data received from the previous sensor(s) and datagenerated by the current sensor, the next sensor can determine alocation of the detected event, e.g., in a field of view of sensor106-14 and not in a field of view of sensor 106-13.

During stage (L), the sensor 106-12 can receive the notification fromthe sensor 106-13 over the network 110. Stage (L) is similar to stage(K). Sensor 106-12 can determine from the notification that sensor106-14 has detected an event and sensor 106-13 does not detect theevent. Sensor 106-12 can generate sensor data and determine that it doesnot detect the fallen tree or another object in its sensor data. Inresponse, sensor 106-12 can transmit the data received from the previoussensor(s) and the data generated by sensor 106-12 to sensor 106-11.

During stage (M), the sensor 106-11 can receive the notification fromthe sensor 106-12 over the network 110. Stage (M) is similar to stages(K) and (L). Similar to previous sensors, the sensor 106-11 candetermine from the notification that sensor 106-14 has detected an eventand sensor 106-13 and 106-12 do not detect the event. In response, thesensor 106-11 can generate sensor data and determine that it does notdetect a similar event of a fallen tree on the dedicated road 109-1.However, sensor 106-11 can determine from its sensor data a detected amoving vehicle, e.g., autonomous vehicle 108-1, on the dedicated road109-1 and calculate an identity product of the detected vehicle inresponse. The sensor 106-11 can also detect railroad characteristicsfrom the sensor data. The railroad characteristics can include, forexample, a detection of an object on the railroad 104, a detection oftrain 102 traveling on the railroad 104, and other railroad detectioninformation.

Based on the detection of the autonomous vehicle 108-1 traversing thededicated road 109-1, the sensor 106-11 can determine an environment ofthe system 101. For example, the sensor 106-11 can determine from thenotification received from the sensor 106-12 that another sensor ahead,e.g., sensor 106-14, has detected an event, e.g., a fallen tree. In someexamples, the sensor 106-11 can determine from the notification receivedfrom the sensor 106-12 of other detected events such as, an object on aparticular lane of the dedicated road 109-1, a train 102 that has fallenonto the dedicated road 109-1, an icy portion of the dedicated road109-1, a traffic jam on the dedicated road 109-1, a vehicular accidenton the dedicated road 109-1, or another type of event. The notificationcan indicate a location of the detected event based on the sensor thatdetected the event and other information.

The sensor 106-11 can determine a distance of the detected event from alocation of the detected vehicle in its field of view. Specifically, thesensor 106-11 can determine a location of the sensor that detected theevent based on a longitudinal order of the sensors along the dedicatedroad 109-1 and distance between each of the sensors. In response, thesensor 106-11 can calculate a distance that the detected autonomousvehicle 108-1 is from the detected event. For example, if the sensor106-11 determines that a spacing between each of the sensors is 10 feetand the sensor 106-14 that detected the event is three sensors down fromits current location, then the sensor 106-11 can determine that thedetected autonomous vehicle 108-1 is approximately thirty feet from thedetected event. The sensor 106-11 can also determine the speed of theautonomous vehicle 108-1. Based on the current speed of the autonomousvehicle 108-1 and the distance of the autonomous vehicle 108-1 to thedetected event, the sensor 106-11 can determine specific actions for thevehicle to take to avoid the detected event.

For example, the actions can include accelerating, changing lanes,stopping, decelerating, turning left, turning right, making a U-turn,and other actions. In this particular example, the autonomous vehicle108-1 can be traveling at 10 miles per hour (MPH) and the sensor 106-11can determine that the autonomous vehicle 108-1 is sufficiently able tostop before reaching the detected event that is thirty feet ahead. Inresponse, the sensor 106-11 can transmit a notification 107 to theautonomous vehicle 108-1 to stop moving over the network 110. The sensor106-11 can transmit another notification to the autonomous vehicle 108-1to continue moving when the sensors monitoring the dedicated road 109-1and the railroad 104 no longer detect the event of the fallen tree. Anabsence of the fallen tree may indicate that workers removed the fallentree from the dedicated lane. In some examples, the autonomous vehicle108-1 can be traveling at 70 MPH and the sensor 106-11 can determinethat the autonomous vehicle 108-1 does not have sufficient stoppingdistance before reaching the detected event that is thirty feet ahead.In response, the sensor 106-11 can transmit a notification to theautonomous vehicle 108-1 to pull off to the side of the dedicated road109-1 to avoid the fallen tree and have ample space to decelerate. Oncethe sensors no longer detect the previously detected event, e.g., thefallen tree is no longer in the dedicated road 109-1, then the sensorscan communicate with one another indicating that the event is no longerdetected.

In response to the detection and communication by the other sensors thatthe previously detected event is no longer detectable, then the sensor106-11 can transmit an additional notification to the autonomous vehicle108-1. The additional notification can indicate, for example, toaccelerate to a desired speed, decelerate to a desired speed, acceleratefor particular period of time, or to return to previous operating speedsfor traversing the dedicated road 109-1. If one or more of the sensorsdetect an additional event that may impeded the traffic on the dedicatedroad 109-1, impede a train 102 on the railroad 104, impede the train 102on the combine roadway, e.g., railroad 104 in the center of thededicated road 109-1, then the sensors 106 can transmit a notificationto each of the other sensors. If the sensors 106 detect one or moreautonomous vehicles on the dedicated road 109-1 along with thenotification of a detected event, then the sensors 106 can notify theone or more autonomous vehicles accordingly.

In some implementations, the autonomous vehicles 108 traversing thededicated road 109-1 can utilize data provided by the sensors monitoringthe dedicated road 109-1 in conjunction with internally generated sensordata. Specifically, the autonomous vehicles 108 can generate sensor datausing its one or more onboard sensors. The sensor data can include, forexample, audio data, video data, LIDAR data, radar data, and other datatypes. The onboard sensor system can utilize the obtained sensor data toidentify objects within a nearby environment of the autonomous vehicles108. In some implementations, the enhanced machine-learning model canutilize the generated sensor data from within the autonomous vehicle 108and data indicative of the vehicle characteristics in addition to thesensor data provided by the sensors monitoring the dedicated road 109-1.The enhanced machine-learning model can receive as input the generatedsensor data from the autonomous vehicle 108's internal sensors, dataindicative of the vehicle characteristics, e.g., using the vehicle'sCANBUS system, and the sensor data provided by the sensors. In response,the enhanced machine-learning model can output a likelihood of adetected event.

In some examples, the enhanced machine-learning model may apply weightsto these inputs. The enhanced machine-learning model may apply moreweight to the sensor data and inputs supplied by the sensors 106-4through 106-17 than the sensor data and inputs generated by theautonomous vehicle 108-1's internal sensors. In some examples, theenhanced machine-learning model may apply more weight to the sensor dataand inputs generated by the autonomous vehicle 108-1's internal sensorsthan the sensor data and inputs supplied by the sensors 106-4 through106-17. While on the dedicated road 109-1, the autonomous vehicle 108-1can rely more heavily on the external sensors than the internal sensors.In some examples, the enhanced machine-learning model may utilize theinputs generated from the autonomous vehicle 108-1's internal sensors asconfirmation of the external sensor's inputs. For example, if theexternal sensors provide sensor data that indicates for the autonomousvehicle 108-1 to accelerate, then the enhanced machine-learning modelcan analyze the sensor data from the autonomous vehicle 108-1's internalsensors to confirm that the area ahead of the autonomous vehicle 108-1is obstacle free.

In some examples, if the external sensors provide sensor data thatindicate the autonomous vehicle 108-1 is to take an action but theenhanced machine-learning model determines the sensor data generated bythe autonomous vehicle 108-1's internal sensors contradicts theinstructed action, then the enhanced machine-learning model can ignorethe external sensors provided action. In some examples, if the enhancedmachine-learning model determines that the external sensor's instructionand the internal sensors sensor data conflicts, then the onboard sensorsystem of the autonomous vehicle can generate and provide a notificationto the sensor that issued the instruction and to the central server 112notifying of conflicted instruction. The sensor can receive thenotification from the autonomous vehicle 108-1's onboard sensor systemand determine a resolution for the conflict with its instruction. Theresolution may include, for example, notifying other sensors of theconflicted instruction, notifying the central server 112 of theconflicted instruction, and determining whether other sensors can detectthe same event as the sensor that instructed the autonomous vehicle108-1 to take an action based on the detected event. In some examples,the sensor can transmit a notification to the other sensors to disregardor delete the previous instruction. Similarly, the central server 112can analyze the notification and determine how to improve the sensors'capabilities.

In response to the enhanced machine-learning model receiving inputsensor data, the enhanced machine-learning model can generate an outputof a likelihood of a detected event. The onboard sensor system canprovide the likelihood of the detected event to the route guidancesystem of the autonomous vehicle 108-1. The route guidance system canreceive the likelihood of the detected event and can produce actions forthe vehicle to take while traversing the path on the dedicated lane.This is similar to stage (D) from FIG. 1A.

In some implementations, a sensor can notify the railroad system and thecentral server 112 in response to detecting an event on the dedicatedroad 109-1 or the railroad 104. Specifically, the sensor can notify therailroad system and central server 112 so these systems are prepared forthe financial loss caused by the event's disruption. For example, inresponse to determining a tree has fallen on the dedicated road 109-1,the sensor 106-14 can transmit a notification to the interface of therailroad system and the central server 112 to warn of disruption. Whenthe dedicated road 109-1 is blocked by a particular event, the sensors106 can instruct the autonomous vehicles 108 to not enter the dedicatedroad 109-1. Preventing autonomous vehicles 108 from entering thededicated road 109-1 ceases profit generation for the dedicated road109-1 because the tolls are collected from these vehicles. Similarly,when the railroad 104 is blocked by a particular event, the railroadsystem can cease the trains from running on the railroad 104. Thisaction also ceases profit generation for the railroad system because therailroad 104 is not being utilized.

However, the system 101 can offset the profits lost when one system isblocked from being used. For example, if the dedicated road 109-1 isblocked by an event for an extended period of time, then the railroadsystem can increase the number of trains the run on the railroad 104during that time to help offset the lost profits due to the lack oftolls being collected. Similarly, if the railroad 104 is blocked by anevent for an extended period of time, then the central server 112 caninstruct the dedicated road 109-1 to increase the toll costs to helpoffset the lost profits from the railroad system not being utilized.Other examples are also possible.

FIG. 1C is another block diagram that illustrates an example of a system103 for monitoring autonomous vehicles traversing a dedicated roadwaythat runs along railroad rights of way. The system 103 is a continuationof systems 100 and 101. Thus, the functions described with respect tosystem 103 can also be performed in systems 100 and 101. Specifically,the system 103 illustrates the autonomous vehicle 108-1 traversing thededicated road 109-1 and ultimately, exiting the dedicated road 109-1,which runs in parallel to the railroad 104. Moreover, the system 103illustrates sensors 106-14 through 106-17 that monitor the vehicles'traversal along the dedicated road 109-1. Similarly, the system 103illustrates sensors 106-18 through 106-21 which monitor the roadway 109.FIG. 1C illustrates various operations in stages (N) through (O), whichcan be performed in the sequence indicated, in another sequence, withadditional stages, or fewer stages. The stages (N) through (O) followthe stages of (J) through (M) of FIG. 1B.

During stage (N), the sensors 106-14 through sensors 106-17 can monitorthe dedicated road 109-1. In some implementations, the sensors 106-14through 106-17 can monitor the dedicated road 109-1 and the railroad104. Stage (N) is similar to stage (J). During stage (N), at least oneof the sensors 106-14 through 106-17 can detect that the autonomousvehicle 108-1 is approaching the end of the dedicated road 109-1.

In some implementations, in response to detecting that the autonomousvehicle 108-1 satisfies a threshold distance from the end of thededicated road 109-1, a sensor can transmit a notification to theautonomous vehicle 108-1 to switch to normal model processing mode. Insome implementations, a sensor can transmit a notification to theautonomous vehicle 108-1 to switch to the normal processing mode inresponse to detecting the autonomous vehicle 108-1 crossing a markerthat signifies an end of the dedicated road 109-1. The marker thatsignifies the end of the dedicated road 109-1 can be a similar markerthat signified the beginning of the dedicated road 109-1.

In some examples, a designer of systems 100, 101, and 103 may designatea threshold distance of 30 feet. The sensors 106-16 and 106-17 may bedesignated by the designer as the sensors to monitor the autonomousvehicles exiting the dedicated road 109-1. The sensors 106-16 and 106-17can monitor a distance the autonomous vehicle 108-1 is located from theend of the designated road 109-1 by generate sensor data and determiningfrom the sensor data, a current distance between the location of theautonomous vehicle 108-1 and the end of the designated road 109-1. Thedistance can include, for example, a straight-line distance and adistance along the dedicated road 109-1 until the marker is met. Thesensors 106-16 and 106-17 can generate sensor data on a frame-by-framebasis to ensure an accuracy in determining when the threshold distanceis met between the location of the autonomous vehicle 108-1 and the endof the dedicated road 109-1. The sensors 106-16 and 106-17 can indicatethe autonomous vehicle 108-1 satisfies the threshold distance when theautonomous vehicle meets or is within the threshold distance.

In some examples, the sensors 106-16 and 106-17 can monitor when theautonomous vehicle 108-1 crosses the marker signifying the end of thededicated road 109-1. The autonomous vehicle 108-1 can be determined tocross the marker when its front tires cross the marker. Alternatively,the autonomous vehicle 108-1 can be determined to cross the marker whenthe entirety of the vehicle has moved past the marker.

In some implementations, the sensors 106-6 and 106-17 can generate anotification to transmit to the autonomous vehicle 108-1 when exitingthe dedicated road 109-1. The notification can include an instruction toswitch from using the enhanced machine-learning model to the one or moretrained machine-learning models. In response to generating theinstruction, at least one of the sensors, e.g., sensors 106-17, cantransmit the generated notification to the onboard sensor processingsystem of the autonomous vehicle 108-1 over the network 110.

During stage (O), the onboard sensor processing system of the autonomousvehicle 108-1 can receive the generated notification. In response toreceiving the generated notification, the onboard sensor system of theautonomous vehicle 108-1 can switch the enhanced machine-learning modelto the normal processing mode. In some implementations, the onboardsensor system of the autonomous vehicle 108-1 can switch the enhancedmachine-learning model to the normal processing mode in response todetecting its exit of the dedicated road 109-1 using one or more of itssensors. For example, in response to detecting the exit of the dedicatedroad 109-1, the onboard sensor system of the autonomous vehicle 108-1can perform at least one of: (i) deactivating the ability for the one ormore trained machine-learning models to receive additional inputsrelated to the sensor data from sensors monitoring the dedicated road109-1, (ii) deleting the enhanced machine-learning model from memory toaccess the one or more trained machine-learning models, (iii) removingthe enhanced machine-learning model from cache memory and storing theenhanced machine-learning model in main memory, and (iv) transmittingthe enhanced machine-learning model to the central server 112 for laterretrieval and removing the enhanced machine-learning model from memory,to name some examples.

In response to switching the enhanced machine-learning model to thenormal processing mode, the onboard sensor system of the autonomousvehicle 108-1 can activate the one or more trained machine-learningmodels for further processing. In some examples, the onboard sensorsystem can insert the one or more trained machine-learning models incache memory to enable accessing the one or more trainedmachine-learning models on a more frequency and rapid basis. In someexamples, the onboard sensor system can request the one or more trainedmachine-learning models from the central server 112. In this example,the onboard sensor system can transmit a request to the central server112 over network 110 for the one or more trained machine-learning modelsand subsequently receive the one or more trained machine-learning modelsin response from the central server 112.

In some implementations, the autonomous vehicle 108-1 can continue totraverse the roadway 109 after exiting the dedicated road 109-1. Theautonomous vehicle 108-1 can traverse the roadway 109 using the sensordata and the one or more trained machine-learning models, as describedwith respect to stages (A) through (E) of FIG. 1A. The autonomousvehicle 108-1 can continue traversing the roadway 109 using its routeguidance system.

FIG. 2 is a block diagram that illustrates an example of components 200of an autonomous vehicle using a normal operating mode and an enhancedoperating mode.

Specifically, the components 200 of autonomous vehicle 201 illustratesvarious operations related to a normal driving mode 202 and railway ROWmode 204, e.g., the enhanced operation mode. In some implementations,the normal driving mode 202 can be activated when the autonomous vehicle201 traverses a main roadway. For example, the autonomous vehicle 201operates in the normal driving mode 202 when traversing the main roadway109. In some implementations, the railway ROW mode 204 can be activatedwhen the autonomous vehicles traverses a dedicated road that runs inparallel to a railroad. For example, the autonomous vehicle 201 operatesin railway ROW mode 204 when traversing the dedicated road 109-1 thatruns in parallel to the railroad 104.

For example, in the normal driving mode 202, the onboard sensor systemof autonomous vehicle 201 can obtain sensor data 206. The sensor data206 can include sensor data generated by one or more sensors onboard theautonomous vehicle 201. The sensor data can include for example, videodata, audio data, LIDAR data, radar data, and other data types. Thesensor data 206 can illustrate an environment proximate to theautonomous vehicle 201 as seen by its sensors. The environment caninclude, for example, a portion of the roadway proximate to theautonomous vehicle 201, traffic signs, traffic lights, various types oflanes, objects in the roadway, weather, railroad, and other information.The sensors can obtain sensor data in a continuous or periodic fashion.This is similar to stage (A) from FIG. 1A.

The onboard sensor system of the autonomous vehicle 201 can obtainvehicle characteristics 208. For example, the onboard sensor system cancommunicate with various device of the autonomous vehicle 201 utilizingthe CANBUS system to obtain the vehicle characteristic information. Thevehicle characteristic information can include, for example, ABS, cruisecontrol, electric power steering, vehicle dynamics, and battery andrecharging systems, to name a few examples. This is similar to stage (A)from FIG. 1A.

In response to obtaining the sensor data 206 and the vehiclecharacteristics 208, the onboard sensor system can provide the sensordata 206 and the vehicle characteristics 208 as input to the one or moretrained machine-learning models 210. The one or more trainedmachine-learning models 210 can process the received inputs through eachof the nodes in the models. The one or more trained machine-learningmodels 210 can receive inputs and generate outputs on a continuous basisor each time new input data is obtained by the sensors. In someexamples, the one or more trained machine-learning models 210 caninclude a recurrent neural network (RNN) model. In some examples, thecentral server 112 can train the one or more RNN machine-learning modelsusing the data stored in the vehicle database 114, the railroad database116, ROW database 118, and other databases that store images utilizedfor object detection. In some examples, the central server 112 caniteratively train the one or more RNN machine-learning models based onfeedback from the sensors monitoring the roadway and sensors onboard theautonomous vehicles. This is similar to stage (B) from FIG. 1A.

In response to providing the sensor data 206 and the vehiclecharacteristics 208 as inputs to the trained machine-learning models,the one or more trained machine-learning models 210 can output alikelihood detection of an event 212. For example, the one or moretrained machine-learning models 210 can output a detection of 2% of adetected object in the proximity of the autonomous vehicle 201.Similarly, the one or more trained machine-learning models 210 can beconfigured to output a classification of one or more objects identifiedin the sensor data 206 and other detected events in the sensor data 206.This is similar to stage (C) from FIG. 1A.

The route guidance system 214 of the autonomous vehicle 201 can receivethe likelihood detection of an event 212 from the one or more trainedmachine-learning models. The route guidance system 214 can receive theinputs from the vehicle characteristics 208. The route guidance system214 can include one or more algorithmic processes that can monitor alocation of a vehicle in real time, e.g., via geographic coordinatesystem (GPS), and map the location of the vehicle on a digital map. Foran autonomous vehicle, the route guidance system can ensure theautonomous vehicle 201 follows a route guidance from an origin locationto a destination location. The route guidance system can produce actions216 for the vehicle to take while traversing the roadway. The actions216 can include, for example, accelerate, change lanes, stop,decelerate, turn left, turn right, U-turn, and other actions. This issimilar to stage (D) from FIG. 1A. The route guidance system candetermine one or more actions for the vehicle to take based on thelikelihood of detection and the vehicle characteristics 208. Forexample, the route guidance system may rely on the vehiclecharacteristics to determine whether the corresponding vehicle iscapable of taking a particular action based on a particular status ofthe vehicle, e.g., current speed, acceleration, temperature of thevehicle, or other.

Similarly, the route guidance system 214 can determine actions 216 forthe autonomous vehicle 201 to make based on the likelihood detection ofevent 212. For example, the route guidance system 214 can ensure theautonomous vehicle 201 avoids a detected object while traversing to thedestination. In this example, the route guidance system 214 can instructthe autonomous vehicle 201 to move in the left lane in response toanalyzing the likelihood detection of event 212 from the one or moretrained machine-learning models 210. This is similar to stage (E) fromFIG. 1A.

In some implementations, the autonomous vehicle 201 can operate in therailway ROW mode 204, e.g., enhanced mode, when the autonomous vehicle201 is instructed to switch to using the enhanced machine-learningmodel. Specifically, the autonomous vehicle 201 can switch to using theenhanced machine-learning model when traversing the dedicated lane thatruns in parallel to the roadway. One or more sensors monitoring thededicated lane can detect the autonomous vehicle 201's entry to thededicated lane and in response, transmit a notification to the onboardsensor system of the autonomous vehicle 201 to switch to using theenhanced machine-learning model.

As the autonomous vehicle 201 traverses the dedicated lane using theenhanced machine-learning model, e.g., under the railroad ROW mode 204,the onboard sensor system of the autonomous vehicle 201 can receive anotification from one or more sensors. Specifically, the onboard sensorsystem of the autonomous vehicle 201 can receive a notification orsensor data from the sensors monitoring the dedicated lane and providethe received sensor data to the enhanced machine-learning model 220model to produce an output. The notification can include sensor data,e.g., video data, LIDAR data, or radar data, or an instruction thatindicates an action for the autonomous vehicle 201 to take. Morespecifically, the action can indicate more detailed characteristics,such as, accelerate for 10 second, accelerate until a target speed ismet, or decelerate until a target speed is met, to name a few examples.

In response to receiving the instruction from one or more of the sensorsmonitoring the dedicated lane, the onboard sensor system can provide thereceived notification as input to the enhanced machine-learning model220. The onboard sensor system can generate sensor data using sensorsinternal to the autonomous vehicle 201 and provide the internallygenerated sensor data as input to the enhanced machine-learning model220. The onboard sensor system can provide the internally generatedsensor as input to the enhanced machine-learning model 220 to enhancethe accuracy of the enhanced machine-learning model 220. For example,the enhanced machine-learning model 220 can rely on sensor data fromsensors onboard the autonomous vehicle and sensor data from sensorsmonitoring the dedicated road. In response, the enhancedmachine-learning model 220 can produce a likelihood of a detected event.The likelihood of a detected event may include, for example, apercentage or statistical likelihood of a detected event or an actionfor the autonomous vehicle 201 to take. The reliance on sensor data fromboth external and internal sensors is beneficial for at least tworeasons.

First, the enhanced machine-learning model 220 can benefit from sensordata that describes an entirety of the dedicated road. The sensor datanow includes not just observations gleaned within proximity of theautonomous vehicle but also observations gleaned from the entirety ofthe roadway. In this manner, the enhanced machine-learning model 220 canproduce improved likelihoods or decisions for the autonomous vehicleusing more informed sensor data. For example, sensor data from thesensors monitoring the roadway can describe an event of an accident 1mile from the location of the autonomous vehicle. The autonomousvehicle's internal sensor data may indicate that no obstacles existwithin close proximity to the autonomous vehicle, and as such, theenhanced machine-learning model will produce a likelihood of noobstacles on the roadway using the internal sensor data alone. As aresult, the route guidance system of the autonomous vehicle willinstruct the autonomous vehicle to continue on the same road. However,with the added benefit of sensor data that describes the entirety of thededicated road, the enhanced machine-learning model can now produce anindication that the autonomous vehicle should navigate a different pathbecause of the obstacle detected one mile ahead. As such, the addedsensor data from the external sensor data improves the enhancedmachine-learning model's decision capabilities and ultimately, enablesthe autonomous vehicle to glean observations from the entirety of thededicated road.

Second, the sensors monitoring the dedicated road can ensure autonomousvehicles traveling the dedicated road make efficient use of thededicated road. These sensors can identify events and other activitiesthat onboard sensors of the autonomous vehicles cannot identify based ontheir viewing distance and/or limited range. As such, the sensors canensure these autonomous vehicles travel an optimum path to theirdestination by informing of events, activities, or obstacles that mayotherwise disrupt their intended path of travel. By doing so, the flowof traffic on the dedicated road can be managed in an orderly andcontrolled manner.

The onboard sensor system of the autonomous vehicle 201 can then providethe output of the enhanced machine-learning model 220 as input to theroute guidance system 222. The route guidance system 222 can determineactions 224 for the autonomous vehicle 201 to make in light of theoutput produced by the enhanced machine-learning model 220. Routeguidance system 222 and action to take 224 is similar to route guidancesystem 214 and the action to take 216.

FIG. 3 is a flow diagram that illustrates an example of a process 300for monitoring autonomous vehicles traversing a dedicated roadway thatruns along railroad rights of way. The sensors, such as sensors 106, anda central server, may perform the process 300.

The central server may receive data from a railroad system that managesa railroad running parallel to a dedicated roadway (302). Specifically,the central server can receive from an interface data from a railroadsystem that manages a railroad running parallel to the dedicatedroadway. The data identifying the railroad system can include data thatidentifies the interface, which may be provided by a client device,computing device, or other. The data from the railroad system caninclude, for example, past schedules of train trips, a number of carsconnected on a train, a time for an upcoming trip of train, anymechanical issues or failures related to train, contact information fora train operator, or dynamic characteristics related to the train, e.g.,train speed, acceleration, and direction of travel, to name a fewexamples. Similarly, the sensors and the central server can transmitrequests to the interface for querying information from the railroadsystem. This information can helpful in assisting the sensors and thecentral server for determining actions for vehicles traversing thededicated roadway to take while traversing the dedicated roadway.

Each sensor from a plurality of sensors is positioned in a fixedlocation relative to the dedicated roadway, and each sensor cancommunicate with a central server. Moreover, each sensor can detect oneor more autonomous vehicles in a first field of view on the dedicatedroadway (304). For example, the plurality of sensors can be positionedlongitudinal to the direction of traffic on the roadway. Each sensor canbe placed in the ground at a predetermined distance apart from oneanother. Additionally, each sensor's field of view can be positionedtowards a segment or area of the roadway to detect and monitor vehicles.Similarly, each sensor's field of view can be positioned to monitorcharacteristics of a railroad that runs in parallel to the dedicatedroadway. For each detected vehicle, the sensors can perform theoperations as described below. A sensor can detect a particular vehiclein its field of view. The sensor can use object detection or some formof classification to detect an object in its field of view.

Each sensor can generate sensor data for the detected autonomous vehiclebased on the detected vehicle on the dedicated roadway and the datareceived at the interface from the railroad system (306). The sensordata can correspond to an identification of a vehicle type,characteristics of detected vehicle or vehicles, vehicular density perunit area, vehicle congestion, vehicle headway, and vehicle dynamics, toname some examples. The identification of the vehicle type cancorrespond to, for example, a truck, a sedan, a minivan, a hatchback, anSVU, and others. The identification of the vehicle type can be based ona size of the vehicle. Characteristics of the vehicle can include, forexample, vehicle color, vehicle size, wheelbase distance, and length,height, and width of vehicle. Vehicular density per unit area cancorrespond to a number of vehicles measured over a particular area intraffic. Vehicular congestion can correspond to a measure of an amountof traffic and movement rate of the traffic in a particular area.Vehicle headway can correspond to a distance between a first and secondvehicle in a transit system measured in time or in distance. Vehicledynamics can include acceleration, deceleration, and velocity of one ormore vehicles traveling along the prior roadways over a period of time.

Each sensor can identify features of the vehicles it detects and can usethe feature data to generate the sensor data. For example, each sensorcan identify features of the detected vehicles that include, forexample, the vehicle color, e.g., as represented by red-green-blue (RGB)characteristics, the vehicle size, e.g., as calculated through opticalcharacteristics, the vehicle class, e.g., as calculated through opticalcharacteristics, and the volume of the vehicle, as calculated throughoptical characteristics. In one such example, a sensor can determinethat a detected vehicle is the color blue, is over 100 ft³ in volume,has a vehicle type of a sedan, and is a medium sized vehicle. Otherexamples are also possible. The sensor can also determine one or morecharacteristics of the vehicle, such as its rate of speed, the distanceaway from the sensor, the vehicle's direction of travel, and a number ofindividuals found in the vehicle, to name a few examples. Based on thegenerated feature data, the sensor can generate sensor data thatincludes an identification of a vehicle type, characteristics ofdetected vehicle or vehicles, vehicular density per unit area, vehiclecongestion, vehicle headway, and vehicle dynamics, to name a fewexamples.

In some implementations, a sensor can query a railroad system forrailroad specific information. This information can include, forexample, characteristics of a train currently traversing the roadway,characteristics of previous trains that have traversed the roadway, andcharacteristics of the railroad, to name some examples. Each sensor canalso query for train and railroad information from the interface thatcommunicates with the railroad system. In some implementations, thecentral server can query for train and railroad information from theinterface that communicates with the railroad system.

In some implementations, each sensor can monitor train activities on therailroad. These sensors may include, for example, omni-directionalcapability that enables viewing and obtaining sensor data from eachdirection simultaneously, in a 360-degree fashion. In this manner, thesensor can not only monitor autonomous vehicles entering, traversing,and exiting the dedicated roadway, but also, monitoring on or moretrains traversing the railroad. The sensors can monitor the sensor dataof the railroad to aid the sensors in determining actions for theautonomous vehicles to take while traversing the dedicated roadway.

Each sensor can generate observational data based on the generatedsensor data (308). For example, when a sensor generates sensor data ofthe feature data, the sensor can generate an identity product of thefeature data and can transmit data representing the identity product ofthe feature data when the corresponding detected vehicle has exited thesensor's field of view. The data representing the identity product ofthe feature data can include, for example, a data structure, a matrix,or a link to data stored in a database. Each sensor can communicate ortransmit the sensor data and observational data to other varioussensors. For example, a sensor that generated sensor data can transmitthe generated sensor data and observational data to the next sensor inthe direction of traffic.

The next sensor can receive the data representing the identity productof the feature data and can compare the data representing the identityproduct of the feature data to new feature data generated by the nextsensor. The next sensor performs this comparison to determine whether itis seeing the same vehicle as seen by the previous sensor, e.g., thesensor that transmitted the data representing the identity product ofthe feature data to the next sensor.

The sensors can also generate observational data that also describeevents occurring on the dedicated roadway. The observational data caninclude, for example, a fallen tree, an obstacle on the dedicatedroadway, an icy portion of the dedicated roadway, a traffic jam, avehicular accident, a train that has fallen on the dedicated roadway, oranother type of event. The observational data can also indicate alocation of the detected event on the dedicated roadway based on thegenerated sensor data. The observational data can be shared betweensensors and shared between sensors and the central server.

Each sensor can instruct the detected autonomous vehicle to switch to anenhanced processing mode (310). In some implementations, the autonomousvehicles that traverse a roadway can receive instructions from sensorsproximate to and monitoring the dedicated roadway to enhance itsthinking. Specifically, the autonomous vehicles can receive instructionsfrom these sensors to switch to using an enhanced processing mode. Theenhanced processing mode is a mode used by the autonomous truck to notonly rely on sensor data generated by sensors onboard the autonomousvehicle but also rely on sensor data or instructions provided by thesensors proximate to the dedicated roadway. These sensors can offerinsights describing events and detection of actors on the dedicatedroadway that may be unseen by the onboard sensors of the autonomousvehicles. When switching to the enhanced processing the mode, theautonomous vehicle can activate an enhanced machine-learning algorithmthat uses both sensor data from onboard sensors and sensor data fromexternal sensors monitoring the dedicated roadway.

In some implementations, the sensors can monitor an autonomous vehicle'sentry into the dedicated road using sensor data. The sensor data canillustrate an autonomous vehicle traversing toward the dedicated roadand upon detecting the autonomous vehicle crossing over a marker, beingwithin a threshold distance from an entrance of the dedicated road, orentering the dedicated roadway, to name a few examples, one or moresensors can transmit a notification to the autonomous vehicle to switchto using the enhanced processing mode. The sensors can transmit anotification to the onboard sensor system of the autonomous vehicle toswitch from using a normal processing mode to using the enhancedprocessing mode in response to detecting the autonomous vehicle's entry.For example, the sensors may utilize the identity product of theautonomous vehicle as a means to detect the vehicle's entry into thededicated lane. The enhanced processing mode is a setting in which theautonomous vehicle provides sensor data from the sensors monitoring thededicated roadway and sensor data from onboard the autonomous vehicle toan enhanced trained machine-learning model. The output of the enhancedtrained machine-learning model can be a likelihood of a detected event.The autonomous vehicle can use the output to determine a path for thevehicle to take while traversing the dedicated roadway.

Each sensor can determine an action for the detected autonomous vehiclebased on the generated observational data, the action indicative of anaction the autonomous vehicle should take when traversing the dedicatedroadway (312). Specifically, each sensor may detect observations fromthe dedicated roadway and observations of one or more trains traversingthe railroad. The sensor data may indicate, for example, a tree hasfallen across the dedicated roadway, a train has derailed off therailroad and landed on the dedicated roadway, an icy patch on thededicated roadway, a traffic jam, traffic congestion, a roadway clear ofobstacles, and data indicative of other events.

The sensors may communicate this information to the central server,where the central server can determine other specific informationrelated to vehicles traversing the roadway. For example, the centralserver may determine the prevailing speeds of autonomous vehiclestraversing the roadway, which can aid in indicating which speedsvehicles should travel along the dedicated roadway. Similarly, thecentral server may determine the vehicle dynamics of vehicles currentlytraversing the roadway, and characteristics of one or more trainscurrently traversing the roadway. Using this information, the centralserver and/or the sensors can determine one or more actions for the oneor more autonomous vehicles traversing the dedicated roadway to take.These actions can include, for example, accelerate, change lanes, stop,decelerate, turn left, turn right, U-turn, and other actions. In someimplementations, the central server may transmit the one or more actionsfor the autonomous vehicles to take to the sensors monitoring the areaproximate to the dedicated roadway.

Each sensor can instruct the autonomous vehicle to traverse thededicated roadway based on the determined action (314). In someimplementations, a sensor can transmit a notification to the autonomousvehicle to take a specific action. The specific action may be an actiongenerated by one or more sensors monitoring the dedicated roadway or anaction generated by the central server. For example, if the sensorsdetect that one or more autonomous vehicles are to potentially collidewith a derailed train on the dedicated roadway, the sensors can transmita notification for the autonomous vehicles to take specific actions.These actions can include, rerouting traffic on the dedicated roadway toavoid the derailed train, decelerating each of the autonomous vehiclesand indicating to change lanes to avoid the derailed train, and stoppingthe autonomous vehicles from colliding with the derailed train, to namea few examples. The sensors can send one or multiple instructions to theautonomous vehicles for actions to take regarding avoiding the derailedtrain.

In some implementations, the sensors can send actions for the autonomousvehicles to take when no event or obstacle is identified on the detectedroadway. These actions can include, for example, accelerating to atarget speed, decelerating to a target speed, remaining in the lane ofthe dedicated roadway, and switching to a normal processing mode uponexiting the dedicated roadway, to name a few examples. The autonomousvehicle's enhanced trained machine-learning model and route guidancesystem can maneuver the autonomous vehicle based on instructionsprovided by the sensors monitoring the dedicated roadway and sensor datagenerated by the sensors onboard the autonomous vehicle.

In some implementations, the sensors monitoring the dedicated roadwaycan determine when the autonomous vehicle is proximate to the end of thededicated roadway. The sensors can determine when the autonomous vehicleis within a threshold distance of the end of the dedicated roadway orhas exited the dedicated roadway. The sensors can perform this functionfor multiple autonomous vehicles traversing the dedicated roadway. Inresponse to detecting the one or more autonomous vehicles beingproximate to the end of the dedicated roadway, the sensors can transmita notification to the autonomous vehicles to switch to the normalprocessing mode. The notification can indicate to these autonomousvehicles exiting the dedicated roadway to switch from the enhanced modeto the normal processing mode.

In the normal processing mode, the autonomous vehicle uses a trainedmachine-learning model that processes sensor data from the onboardsensors. Moreover, the trained machine-learning model does not usesensor data or instructions as input from the sensors monitoring thededicated roadway because the autonomous vehicle is no longer travelingthe dedicated roadway. Generally, these sensors monitoring the dedicatedroadway only communicate with autonomous vehicles traversing thededicated roadway. When the autonomous vehicles exit the dedicatedroadway, they no longer need to communicate with these sensors thatmonitor the dedicated roadway. As such, the autonomous vehicle uses thetrained machine-learning model in the normal processing mode prior tothe entrance of the dedicated roadway and after exiting the dedicatedroadway.

Embodiments of the invention and all of the functional operationsdescribed in this specification may be implemented in digital electroniccircuitry, or in computer software, firmware, or hardware, including thestructures disclosed in this specification and their structuralequivalents, or in combinations of one or more of them. Embodiments ofthe invention may be implemented as one or more computer programproducts, i.e., one or more modules of computer program instructionsencoded on a computer-readable medium for execution by, or to controlthe operation of, data processing apparatus. The computer readablemedium may be a non-transitory computer readable storage medium, amachine-readable storage device, a machine-readable storage substrate, amemory device, a composition of matter effecting a machine-readablepropagated signal, or a combination of one or more of them. The term“data processing apparatus” encompasses all apparatus, devices, andmachines for processing data, including by way of example a programmableprocessor, a computer, or multiple processors or computers. Theapparatus may include, in addition to hardware, code that creates anexecution environment for the computer program in question, e.g., codethat constitutes processor firmware, a protocol stack, a databasemanagement system, an operating system, or a combination of one or moreof them. A propagated signal is an artificially generated signal, e.g.,a machine-generated electrical, optical, or electromagnetic signal thatis generated to encode information for transmission to suitable receiverapparatus.

A computer program (also known as a program, software, softwareapplication, script, or code) may be written in any form of programminglanguage, including compiled or interpreted languages, and it may bedeployed in any form, including as a stand-alone program or as a module,component, subroutine, or other unit suitable for use in a computingenvironment. A computer program does not necessarily correspond to afile in a file system. A program may be stored in a portion of a filethat holds other programs or data (e.g., one or more scripts stored in amarkup language document), in a single file dedicated to the program inquestion, or in multiple coordinated files (e.g., files that store oneor more modules, sub programs, or portions of code). A computer programmay be deployed to be executed on one computer or on multiple computersthat are located at one site or distributed across multiple sites andinterconnected by a communication network.

The processes and logic flows described in this specification may beperformed by one or more programmable processors executing one or morecomputer programs to perform functions by operating on input data andgenerating output. The processes and logic flows may also be performedby, and apparatus may also be implemented as, special purpose logiccircuitry, e.g., an FPGA (field programmable gate array) or an ASIC(application specific integrated circuit).

Processors suitable for the execution of a computer program include, byway of example, both general and special purpose microprocessors, andany one or more processors of any kind of digital computer. Generally, aprocessor will receive instructions and data from a read only memory ora random access memory or both. The essential elements of a computer area processor for performing instructions and one or more memory devicesfor storing instructions and data. Generally, a computer will alsoinclude, or be operatively coupled to receive data from or transfer datato, or both, one or more mass storage devices for storing data, e.g.,magnetic, magneto optical disks, or optical disks. However, a computerneed not have such devices. Moreover, a computer may be embedded inanother device, e.g., a tablet computer, a mobile telephone, a personaldigital assistant (PDA), a mobile audio player, a Global PositioningSystem (GPS) receiver, to name just a few. Computer readable mediasuitable for storing computer program instructions and data include allforms of non-volatile memory, media, and memory devices, including byway of example semiconductor memory devices, e.g., EPROM, EEPROM, andflash memory devices; magnetic disks, e.g., internal hard disks orremovable disks; magneto optical disks; and CD ROM and DVD-ROM disks.The processor and the memory may be supplemented by, or incorporated in,special purpose logic circuitry.

To provide for interaction with a user, embodiments of the invention maybe implemented on a computer having a display device, e.g., a CRT(cathode ray tube) or LCD (liquid crystal display) monitor, fordisplaying information to the user and a keyboard and a pointing device,e.g., a mouse or a trackball, by which the user may provide input to thecomputer. Other kinds of devices may be used to provide for interactionwith a user as well; for example, feedback provided to the user may beany form of sensory feedback, e.g., visual feedback, auditory feedback,or tactile feedback; and input from the user may be received in anyform, including acoustic, speech, or tactile input.

Embodiments of the invention may be implemented in a computing systemthat includes a back end component, e.g., as a data server, or thatincludes a middleware component, e.g., an application server, or thatincludes a front end component, e.g., a client computer having agraphical user interface or a Web browser through which a user mayinteract with an implementation of the invention, or any combination ofone or more such back end, middleware, or front end components. Thecomponents of the system may be interconnected by any form or medium ofdigital data communication, e.g., a communication network. Examples ofcommunication networks include a local area network (“LAN”) and a widearea network (“WAN”), e.g., the Internet.

The computing system may include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other.

Although a few implementations have been described in detail above,other modifications are possible. For example, while a clientapplication is described as accessing the delegate(s), in otherimplementations the delegate(s) may be employed by other applicationsimplemented by one or more processors, such as an application executingon one or more servers. In addition, the logic flows depicted in thefigures do not require the particular order shown, or sequential order,to achieve desirable results. In addition, other actions may beprovided, or actions may be eliminated, from the described flows, andother components may be added to, or removed from, the describedsystems. Accordingly, other implementations are within the scope of thefollowing claims.

While this specification contains many specific implementation details,these should not be construed as limitations on the scope of anyinvention or of what may be claimed, but rather as descriptions offeatures that may be specific to particular embodiments of particularinventions. Certain features that are described in this specification inthe context of separate embodiments can also be implemented incombination in a single embodiment. Conversely, various features thatare described in the context of a single embodiment can also beimplemented in multiple embodiments separately or in any suitablesubcombination. Moreover, although features may be described above asacting in certain combinations and even initially claimed as such, oneor more features from a claimed combination can in some cases be excisedfrom the combination, and the claimed combination may be directed to asubcombination or variation of a subcombination.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults. In certain circumstances, multitasking and parallel processingmay be advantageous. Moreover, the separation of various system modulesand components in the embodiments described above should not beunderstood as requiring such separation in all embodiments, and itshould be understood that the described program components and systemscan generally be integrated together in a single software product orpackaged into multiple software products.

Particular embodiments of the subject matter have been described. Otherembodiments are within the scope of the following claims. For example,the actions recited in the claims can be performed in a different orderand still achieve desirable results. As one example, the processesdepicted in the accompanying figures do not necessarily require theparticular order shown, or sequential order, to achieve desirableresults. In certain implementations, multitasking and parallelprocessing may be advantageous.

What is claimed is:
 1. A system comprising: a central server; aninterface for receiving data from a railroad system that manages arailroad running parallel to a dedicated roadway; a plurality of sensorspositioned in a fixed location relative to the dedicated roadway,wherein each sensor in the plurality of sensors: detects one or moreautonomous vehicles in a first field of view on the dedicated roadway,and for each detected autonomous vehicle: generates sensor data for thedetected autonomous vehicle based on the detected autonomous vehicle onthe dedicated roadway and the data received at the interface from therailroad system; generates observational data based on the generatedsensor data; instructs the detected autonomous vehicle to switch to anenhanced processing mode based on the generated observational data;determines an action for the detected autonomous vehicle based on thegenerated observational data, the action indicative of an action theautonomous vehicle should take when traversing the dedicated roadway;and instructs the detected autonomous vehicle to traverse the dedicatedroadway based on the determined action; acquires additional sensor dataof the autonomous vehicles traversing the dedicated roadway; detects anidentity for each of the autonomous vehicles from the additional sensordata; determines a location for at least some of the autonomous vehicleson the dedicated roadway; and from the identity for each of theautonomous vehicles, determines that the at least some of the autonomousvehicles are proximate to the end of the dedicated roadway, and inresponse, transmits an indication to the at least some of the autonomousvehicles to switch to a normal processing mode.
 2. The system of claim1, wherein the interface displays data related to the railroad thattraverses in parallel to the dedicated roadway and one or more trainstraverse the railroad, the data comprising a number of the one or moretrains, a direction of the one or more trains traveling on the railroad,and a number of railroads.
 3. The system of claim 1, wherein theautonomous vehicles that traverse the dedicated roadway compriseautonomous trucks.
 4. The system of claim 1, wherein: the plurality ofsensors: acquires first sensor data of the autonomous vehiclestraversing the dedicated roadway; detects an identity for each of theautonomous vehicles from the first sensor data; from the identity foreach of the autonomous vehicles, determines that each of the autonomousvehicles have entered the dedicated roadway; and in response, transmitsan indication to each of the autonomous vehicles to switch to theenhanced processing mode.
 5. The system of claim 4, wherein the enhancedprocessing mode comprises (i) a first setting for operating the detectedautonomous vehicle using sensor data from the plurality of sensors andthe sensor data onboard the detected autonomous vehicle or (ii) a secondsetting in which the detected autonomous vehicle utilizes an enhancedtrained machine-learning model for producing actions for traversing thededicated roadway.
 6. The system of claim 1, wherein: the plurality ofsensors: acquires second sensor data of the autonomous vehiclestraversing the dedicated roadway; acquires third sensor data of one ormore trains traversing the railroad that traverses in parallel to thededicated roadway; and transmits the acquired second and third sensordata to the central server; wherein the central server: receives thesecond and third sensor data from each sensor of the plurality ofsensors: determines from the received second and third sensor data:prevailing speeds of the autonomous vehicles traversing the dedicatedroadway; vehicle dynamics of the autonomous vehicles traversing thededicated roadway; objects currently identified on the dedicatedroadway; and characteristics of the one or more trains traversing therailroad; and in response, determines one or more actions for each ofthe autonomous vehicles for traversing the dedicated roadway based onthe prevailing speeds, the vehicle dynamics, the objects currentlyidentified, and the characteristics of the one or more trains traversingthe railroad.
 7. The system of claim 1, wherein: the plurality ofsensors: acquires fourth sensor data of the autonomous vehiclestraversing the dedicated roadway; acquires fifth sensor data indicativeof a train that has derailed off the railroad, the railroad traversingin parallel to the dedicated the roadway; transmits the acquired fourthand fifth sensor data to the central server; wherein the central server:receives the acquired fourth and fifth sensor data from each sensor ofthe plurality of sensors: determines from the received fourth and fifthsensor data: a first indication that the train has derailed off therailroad; a second indication that at least some of the autonomousvehicles traversing the dedicated roadway are on a path to collide withthe derailed train; and in response, transmits an instruction to the atleast some of the autonomous vehicles to (i) reroute traffic on thededicated roadway to avoid the derailed train, (ii) decelerate theautonomous vehicles, (iii) stop the autonomous vehicles from collidingwith the derailed train, or (iv) a combination of (i)-(iii).
 8. Thesystem of claim 1, wherein the normal processing mode comprises asetting for operating an autonomous vehicle with an onboard trainedmachine-learning model used (i) prior to entrance of the autonomousvehicle to the dedicated roadway and (ii) after the autonomous vehicleexits the dedicated roadway.
 9. A computer-implemented methodcomprising: receiving, at an interface, data from a railroad system thatmanages a railroad running parallel to a dedicated roadway; detecting,by each sensor in a plurality of sensors positioned in a fixed locationrelative to the dedicated roadway, one or more autonomous vehicles in afirst field of view on the dedicated roadway, and for each detectedautonomous vehicle: generates sensor data for the detected autonomousvehicle based on the detected autonomous vehicle on the dedicatedroadway and the data received at the interface from the railroad system;generates observational data based on the generated sensor data;instructs the detected autonomous vehicle to switch to an enhancedprocessing mode based on the generated observational data; determines anaction for the detected autonomous vehicle based on the generatedobservational data, the action indicative of an action the autonomousvehicle should take when traversing the dedicated roadway; and instructsthe detected autonomous vehicle to traverse the dedicated roadway basedon the determined action; acquiring additional sensor data of theautonomous vehicles traversing the dedicated roadway; detecting anidentity for each of the autonomous vehicles from the additional sensordata; determining a location for at least some of the autonomousvehicles on the dedicated roadway; and from the identity for each of theautonomous vehicles, determines that the at least some of the autonomousvehicles are proximate to the end of the dedicated roadway, and inresponse, transmitting an indication to the at least some of theautonomous vehicles to switch to a normal processing mode.
 10. Thecomputer-implemented method of claim 9, further comprising: displaying,at the interface, data related to the railroad that traverses inparallel to the dedicated roadway and one or more trains traverse therailroad, the data comprising a number of the one or more trains, adirection of the one or more trains traveling on the railroad, and anumber of railroads.
 11. The computer-implemented method of claim 9,wherein the autonomous vehicles that traverse the dedicated roadwaycomprise autonomous trucks.
 12. The computer-implemented method of claim9, further comprising: acquiring, by the plurality of sensors, firstsensor data of the autonomous vehicles traversing the dedicated roadway;detecting, by the plurality of sensors, an identity for each of theautonomous vehicles from the first sensor data; from the identity foreach of the autonomous vehicles, determining, by the plurality ofsensors, that each of the autonomous vehicles have entered the dedicatedroadway; and in response, transmitting, by the plurality of sensors, anindication to each of the autonomous vehicles to switch to the enhancedprocessing mode.
 13. The computer-implemented method of claim 12,wherein the enhanced processing mode comprises (i) a first setting foroperating the detected vehicle using sensor data from the plurality ofsensors and the sensor data onboard the autonomous vehicle or (ii) asecond setting in which the detected autonomous vehicle utilizes anenhanced trained machine-learning model for producing actions fortraversing the dedicated roadway.
 14. The computer-implemented method ofclaim 9, further comprising: acquiring, by the plurality of sensors,second sensor data of the autonomous vehicles traversing the dedicatedroadway; acquiring, by the plurality of sensors, third sensor data ofone or more trains traversing the railroad that traverses in parallel tothe dedicated roadway; transmitting, the plurality of sensors, theacquired second and third sensor data to a central server; receiving, bythe central server, the second and third sensor data from each sensor ofthe plurality of sensors: determining, by the central server, from thereceived second and third sensor data: prevailing speeds of theautonomous vehicles traversing the dedicated roadway; vehicle dynamicsof the autonomous vehicles traversing the dedicated roadway; objectscurrently identified on the dedicated roadway; and characteristics ofthe one or more trains traversing the railroad; and in response,determining, by the central server, one or more actions for each of theautonomous vehicles for traversing the dedicated roadway based on theprevailing speeds, the vehicle dynamics, the objects currentlyidentified, and the characteristics of the one or more trains traversingthe railroad.
 15. The computer-implemented method of claim 9, furthercomprising: acquiring, by the plurality of sensors, fourth sensor dataof the autonomous vehicles traversing the dedicated roadway; acquiring,by the plurality of sensors, fifth sensor data indicative of a trainthat has derailed off the railroad, the railroad traversing in parallelto the dedicated roadway; transmitting, by the plurality of sensors, theacquired fourth and fifth sensor data to a central server; receiving, bythe central server, the acquired fourth and fifth sensor data from eachsensor of the plurality of sensors: determining, by the central server,from the received fourth and fifth sensor data: a first indication thatthe train has derailed off the railroad; a second indication that atleast some of the autonomous vehicles traversing the dedicated roadwayare on a path to collide with the derailed train; and in response,transmitting, by the central server, an instruction to the at least someof the autonomous vehicles to (i) reroute traffic on the dedicatedroadway to avoid the derailed train, (ii) decelerate the autonomousvehicles, (iii) stop the autonomous vehicles from colliding with thederailed train, or (iv) a combination of (i)-(iii).
 16. Thecomputer-implemented method of claim 9, wherein the normal processingmode comprises a setting for operating an autonomous vehicle with anonboard trained machine-learning model used (i) prior to entrance of theautonomous vehicle to the dedicated roadway and (ii) after theautonomous vehicle exits the dedicated roadway.
 17. One or morenon-transitory machine-readable media storing instructions that, whenexecuted by one or more processing devices, cause the one or moreprocessing devices to perform operations comprising: receiving, at aninterface, data from a railroad system that manages a railroad runningparallel to a dedicated roadway; detecting, by each sensor in aplurality of sensors positioned in a fixed location relative to thededicated roadway, one or more autonomous vehicles in a first field ofview on the dedicated roadway, and for each detected autonomous vehicle:generates sensor data for the detected autonomous vehicle based on thedetected autonomous vehicle on the dedicated roadway and the datareceived at the interface from the railroad system; generatesobservational data based on the generated sensor data; instructs thedetected autonomous vehicle to switch to an enhanced processing modebased on the generated observational data; determines an action for thedetected autonomous vehicle based on the generated observational data,the action indicative of an action the autonomous vehicle should takewhen traversing the dedicated roadway; and instructs the detectedautonomous vehicle to traverse the dedicated roadway based on thedetermined action; acquiring additional sensor data of the autonomousvehicles traversing the dedicated roadway; detecting an identity foreach of the autonomous vehicles from the additional sensor data;determining a location for at least some of the autonomous vehicles onthe dedicated roadway; and from the identity for each of the autonomousvehicles, determines that the at least some of the autonomous vehiclesare proximate to the end of the dedicated roadway, and in response,transmitting an indication to the at least some of the autonomousvehicles to switch to a normal processing mode.
 18. The one or morenon-transitory machine-readable media of claim 17, further comprising:displaying, at the interface, data related to the railroad thattraverses in parallel to the dedicated roadway and one or more trainstraverse the railroad, the data comprising a number of the one or moretrains, a direction of the one or more trains traveling on the railroad,and a number of railroads.